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ABSTRACT 


Today’s  first  responders  confront  a  common  challenge,  namely  the  lack  of 
exposure  to  and  experience  with  asymmetric  threats  (i.e.,  terrorism  and  natural  disasters) 
in  training  venues  that  would  enable  them  to  develop  familiarity  with  these  novel 
situations.  Different  problem-solving  strategies  currently  employed  by  today’s  first 
responders  are  described,  along  with  situation  awareness  and  how  to  best  leverage  first- 
responder  experience.  Literature  on  expert  versus  novice  decision  making,  situation 
awareness,  recognition-primed  decision  making,  and  scenario-based  learning  was 
leveraged  to  design  the  thesis  experiment.  Through  scenario-based  exercises,  the  thesis 
attempted  to  discover  whether  the  decision-making  skills  of  an  experienced  fire  officer 
(expert)  can  be  learned  by  newly  promoted  officers  (novice).  Results  from  this 
experiment  provided  insight  and  plausible  remedies  regarding  today’s  asymmetric  threats 
in  the  form  of  recommendations  to  enhance  the  first  responder’s  ability  to  develop  good 
situational  awareness  and  decision  making 

The  goal  now  is  to  use  research  results  and  recommendations  as  a  springboard  to 
develop  training  that  helps  a  novice  to  effectively  respond  to  asymmetric  threats. 
Experiment  results  indicate  that,  by  combining  scenarios  designed  to  expose  novices  to 
situations  they  may  not  experience  during  routine  operations  with  timely  expert  feedback, 
an  individual’s  decision-making  skills  and  situation  awareness  can  be  improved. 
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I.  INTRODUCTION 


Naturalistic  decision  making  is  the  way  individuals  use  their  experience  to  solve 
problems  in  real-world  settings,  identifying  their  situation  and  taking  action  during  times 
of  uncertainty.  Decision-making  theorist  Gary  Klein  believes  that  today’s  first  responders 
often  use  their  “power  of  intuition”  when  problem  solving  in  their  respective  lines  of 
work.  They  have  to  —  time  is  a  rare  commodity  at  a  fire,  in  an  emergency  room,  or  at  a 
crime  scene.  Experienced  firefighters,  nurses,  and  police  officers  exhibit  expert  problem¬ 
solving  skills  every  day  in  their  emergency  response  roles. 

Thesis  research  —  and  beliefs  regarding  naturalistic  decision  making, 
expert/novice  problem  solving,  and  situation  awareness  —  remains  loyal  to  those  of 
Klein,  Endsley,  and  Cohen.  Namely,  that  effective  first-responder  problem  solving  is 
often  a  product  of  experience.  While  first  responders  have  been  interviewed  and  studied 
regarding  their  decisions  making  experience  and  processes  employed,  this  thesis  connects 
decision-making  theory  with  current  novel  real-world  problem  scenarios  by  identifying 
the  problems  facing  today’s  first  responders,  plausible  solutions,  and  recommended 
treatments  for  training  personnel  on  novel  situations. 

This  research  was  conducted  for  several  reasons.  First,  it  joins  the  beliefs  of 
today’s  leading  decision-making  theorist,  Gary  Klein  with  first-responder  problem 
solving  via  real-world  examples  and  scenarios.  Second,  through  the  use  of  real-world 
examples  derived  from  personal  experiences  and  shared  story  telling,  an  individual  often 
sees  the  relevance  of  the  theory  to  their  particular  domain.  Third,  when  the  connection 
between  theory  and  reality  is  discovered,  the  learning  has  meaning.  For  instance,  what 
does  an  expert  consider  and  what  analogues  does  he  create  when  solving  present-day 
problems?  These  considerations  and  analogues  are  captured  through  this  thesis’  scenario- 
based  examples  and  experiments. 


An  expert  and  novice  fire  officer  may  intuitively  agree  on  the  risk  vs.  reward 
factors  regarding  the  commitment  of  firefighting  personnel  to  a  vacant  building  fire. 
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When  problems  become  complex,  however,  resulting  in  decision-making  situations  that 
require  more  than  basic  knowledge  to  solve,  individuals  must  rely  on  more  advanced 
skills  to  help  them  reach  a  decision.  For  example,  when  the  officer  in  command  receives 
reports  that  the  building  is  now  in  danger  of  falling  down  and  that  squatters  have  been 
living  in  the  building,  the  risk  vs.  reward  factors  become  more  complex. 


By  providing  exposure  to  the  skilled  performance  that  reflects  experts’  intuitive 
thought  processes  during  everyday  emergencies,  today’s  less-experienced  decision 
makers  can  begin  acquiring  the  same  skills  through  structured  scenario-based  training. 
When  expert  feedback  on  their  responses  to  asymmetric  scenarios  is  provided  to  novices, 
the  novices  can  begin  to  leam  how  to  think  through  a  problem  in  a  way  that  is  similar  to 
the  experts.  Only  then  can  one  attempt  to  take  those  expert  skills  and  talents  and  try  to 
superimpose  them  onto  a  novice’s  problem-solving  skills  in  an  asymmetric  environment. 

A.  PROBLEM  STATEMENT 

Fire  service  personnel  solve  fire  ground  problems  using  several  decision-making 
strategies;  one  approach  employs  the  recognition-primed  decision-making  (RPD) 
strategy.  RPD  includes  several  techniques:  pattern  matching,  cue  recognition,  mental 
simulations,  and  mental  models.  Pattern  matching  helps  fire  personnel  recognize 
similarities  to  other  situations  by  matching  cues  from  the  current  situation  such  as  the 
color  of  smoke  with  templates  stored  in  memory  and  finding  a  match.  A  template  is  a 
group  of  cues,  stored  in  long-term  memory  that  represents  a  particular  instance  of  a 
situation,  for  example,  a  specific  type  of  fire.  Cue  recognitions  help  the  decision  maker  to 
identity  critical  pieces  in  the  patterns.  Mental  simulation  allows  the  decision  maker  to 
mentally  simulate  applying  a  particular  course  of  action  to  see  how  well  it  will  work.1 
Mental  models  store  information  in  an  organized  fashion,  and  are  often  referred  to  as  the 
decision  maker’s  “schema.”2  Mental  models  afford  decision  makers  the  ability  to 

1  Gary  Klein,  Sources  of  Power:  How  People  Make  Decisions  (Cambridge,  Mass:  M.I.T.  Press,  1998), 
92. 

2Mica  R.  Endsley,  “Expertise  and  Situation  Awareness”  (eds)  Neil  Chamess  et  al.,  The  Cambridge 
Handbook  of  Expertise  and  Expert  Performance  (New  York:  Cambridge  University  Press,  2006):  638. 
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construct  personal  paradigms  (the  individual  lens  we  use  to  view  the  world)  that  can 
accommodate  the  current  dynamic  event.  Many  first  responder  decision-making 
techniques  are  learned  through  trial-and-error  problem  solving  during  emergency 
response  events.  Once  these  techniques  are  mastered,  fire  service  personnel  use  them  as 
decision-making  tools  to  effectively  assist  them  in  their  problem-solving  strategies  during 
an  emergency  response  operation. 

A  second  strategy  uses  fire  service  Standard  Operating  Procedures  (SOPs).  SOPs 
are  developed  from  fire  service  training  guides  and  related  publication  manuals.  The 
procedures  included  in  training  guides  are  the  result  of  over  one  hundred  years  of 
collective  on-the-job  experience,  and  are  reinforced  through  training  and  exercises.  For 
example,  an  effective  method  to  evaluate  a  fire  department’s  SOPs  and  operational 
readiness  concerning  a  high-rise  building  fire  is  to  conduct  a  full-scale  exercise  in  a 
building  of  this  type.  From  a  decision-making  standpoint,  SOPs  are  imperative  because 
fire  service  personnel  from  a  variety  of  locals  often  respond  to  the  same  incident;  to  work 
together  effectively,  they  need  common  points  of  reference  and  similar  training 
backgrounds.  The  National  Incident  Management  System  (NIMS)  addressed  multi¬ 
department  operations  by  introducing  an  incident  management  system  that  provides  a 
consistent  approach  for  all  levels  of  response  to  work  together  in  preparing  for, 
responding  to,  and  recovering  from  domestic  incidents  regardless  of  size  or  complexity.3 
New  York  City’s  Citywide  Incident  Management  System  (CIMS)  is  the  City’s 
implementation  of  NIMS  for  managing  emergency  incidents  and  planned  events  in  New 
York  City.4  For  example,  an  incident  commander’s  operational  decision  needs  to  be 
understood,  not  only  at  that  strategic  level,  but  also  at  the  tactical  level.  Because  strategy 
and  tactics  are  somewhat  streamlined  throughout  the  fire  service,  decisions  made  and 
problems  solved  will  be  predicated  on  the  common  goal  of  protecting  life  and  saving 
property. 


3  Department  of  Homeland  Security,  National  Incident  Management  System  (March  1,  2004):  1. 

4  New  York  City  Office  of  Emergency  Management,  Citywide  Incident  Management  System  (April  6, 
2005):  6. 
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In  their  attempt  to  solve  problems  and  make  effective  decisions  under  emergency 
conditions,  fire  service  personnel  synthesize  their  own  real-life  firefighting  expertise  with 
the  collective  experiences  of  numerous  fire  service  personnel  conveyed  via  SOPs, 
training,  and  education.  Experience,  developed  through  direct  participation  and 
observation,  has  given  the  fire  service  a  distinct  advantage  when  it  is  required  to  solve 
problems  in  their  area  of  domain-specificity. 

In  addition  to  their  everyday  problem-solving  and  decision-making  responsibilities, 
fire  service  personnel  are  now  confronting  a  novel  concern:  decision  making  in  an 
asymmetric  threat  environment,  or  situations  with  environmental  factors  that  are  unorthodox, 
surprising,  urgent,  and  unforeseen.  The  decision  strategies  normally  employed  by  fire 
personnel  are  not  always  effective  because  these  novel  situations  do  not  match  the  stored 
patterns  the  firefighter  has  accumulated  from  years  of  experience.  Pattern  matching,  cue 
recognition,  and  mental  simulations  do  not  always  apply  to  these  unique  situations.  Examples 
of  an  asymmetric  threat  environment  include  chemical,  radiological,  biological,  and  other 
forms  of  terrorist  attacks  on  critical  infrastructure  (such  as  9/11),  and  natural  disasters 
(wildland  fires,  hurricanes,  earthquakes,  and  floods)  that  threaten  a  population  that  has  grown 
in  density  over  the  past  fifty  years,  and  thus  are  of  much  greater  consequence  than  those  for 
which  fire  services  were  traditionally  trained  and  prepared.  Urbanization  of  the  rural 
landscape  has  changed  the  way  forest  fires  are  fought.  Trees  are  no  longer  the  resources  most 
severely  exposed  to  fire,  now  it  is  the  homes  and  the  people  living  in  the  forest. 5  As  James 
Davis  has  noted,  ‘Tire  managers  across  the  nation  are  confronting  the  rapidly  developing 
problem  of  people  moving  into  wildland  areas,  increasing  what  has  been  termed  the 

wildland-urban  interface.”^ 

Most  significant  fire-related  events  have  the  potential  to  generate  many  casualties 
and  cause  widespread  environmental  damage.  An  asymmetric  attack  is  not  inherently 
worse,  but  is  rather  unconventional  in  some  aspect,  obliging  responders  to  operate  in  a 
world  of  uncertainty.  The  fire  service  now  confronts  a  new  threat  without  the  benefit  of 

^  WUI  Professional  Development  Program,  “Module  1:  Trainer’s  Guide  Wildland-Urban  Interface 
Issues  and  Connections.”  http://www.interfacesouth.org/products/pdf/modl.pdf.  [Accessed  June  8,  2007]. 

6  James  B.  Davis,  “The  Wild  land-Urban  Interface:  What  It  Is,  Where  It  Is,  And  Its  Fire  Management 
Problems”  (n.d.):  160.  http://www.cnr.berkelev.edu/~fbeall/050.pdf.  [Accessed  June  8,  2007]. 
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the  real-life  experience  and  the  SOPs  necessary  to  create  effective  situational  awareness 
(SA).  “Simply  put,  SA  is  knowing  what  is  going  on  around  you.  Inherent  in  this 
definition  is  a  notion  of  what  is  important.”  7 


Expert  decision  makers  are  said  to  think  and  act  intuitively,  often  relying  on  “gut 
instinct”  —  honed  by  experience  —  to  make  crucial  decisions.  This  is  naturalistic  decision 
making  in  its  most  basic  form.  Without  experience,  though,  decision  makers  may  reason  like 
novices.  Because  they  have  no  mental  models  stored  (based  on  experience),  there  are  no 
patterns  to  match  or  cues  to  recognize.  In  some  cases,  the  decision  maker  relies  on  a  rational 
choice  strategy.  These  strategies  may  not  produce  the  best  course  of  action  (COA)  for  first 
responder  decision  makers  to  use  in  emergency  situations.  The  problem  with  rational  choice 
strategies  is  that,  because  they  involve  methodical,  multi-step  processes,  they  are  labor-and 
time-intensive;  because  of  this,  the  decisions  are  often  obsolete  and  ineffective  when  it  comes 
time  to  institute  them.  The  following  seven-step  problem-solving  model  from  William 
Duggan’s  “Coup  D’oeil;  Strategic  Intuition  in  Army  Planning.”  is  one  such  example  of  this 
lengthy  process. 
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1  -  ID  the  problem 


2  -  Gather  information 


3  -  Develop  criteria 


4  -  Generate  possible  solutions 


5  -  Analyze  possible  solutions 


6  -  Compare  possible  solutions 


7  -  Make  and  implement  the  decision 


Who,  what,  when,  where  and  why? 


Facts,  assumptions,  and  interests 

J  LINKED  J 

Screening  &  Evaluation 


Suitable,  feasible,  acceptable, 
distinguishable,  and  complete 


Benchmark  { Does  the  solution  achieve 
the  desired  state?) 


Determine  the  best  solution 


Decide  and  act 


Figure  1.  Seven-Step  Problem  Solving  Model  (from  8) 


7  Endsley  “Theoretical  Underpinnings  of  Situational  Awareness:  A  Critical  Review Situation 
Awareness  Analysis  and  Measurement  (Mahwah,  N.J.:Lawrence  Erlbaum  Associates,  2000):  2. 
http://www.satechnologies.com/papers/pdf/satheorychapter.pdf  [Accessed  April  2007]. 

8  William  Duggan,  “ Coup  D  ’oeil:  Strategic  Intuition  in  Army  Planning”  (November  2005):  15. 
http://www.StrategicStudiesInstitute.army.mil/  [Accessed  January  3,  2007]. 
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By  relying  on  a  rational  thought  process,  Klein  suggests  that  inexperienced 
decision  makers  are  viewing  the  problem  via  a  very  specific  narrow  lens.  Klein,  a  leader 
in  the  study  of  problem  solving  and  decision  making,  calls  this  narrow  vision  approach 
“hyperrationality,”  because  decision  makers  try  to  solve  all  problems  based  on  logical 
and  analytical  forms  of  reasoning.9  Decision  makers  try  to  do  all  their  thinking  just  by 
applying  a  rational  procedure.  Imagine  the  ramifications  of  incorporating  a  narrow  lens 
approach  while  addressing  a  homeland  security  prevention,  preparedness,  or  response 
related  matter  and  not  seeing  the  application  process  in  its  entirety.  Envision  a  homeland 
security  prevention  initiative  that  was  not  based  on  experience  in  the  decision-making 
process.  Prevention  planners  use  past  events  as  starting  points  for  future  plans.  By  not 
utilizing  their  previous  experience  as  a  foundation  for  making  decisions,  planners  are  at 
risk  of  missing  the  “big  picture.” 

Many  real-world  events  do  not  allow  for  an  in-depth  analysis  and  size-up  of  the 
scene.  A  fire  officer  operating  at  an  expanding  fire  using  a  rational  choice  strategy  would 
take  so  much  time  to  finally  decide  on  ordering  an  additional  alarm  response  that  the 
level  of  response  would  no  longer  be  adequate  due  to  the  speed  of  the  spreading  fire. 
Instead,  when  responding  to  these  events,  firefighters  frequently  rely  on  spontaneous 
decision-making  and  problem-solving  skills  based  on  many  years  of  accumulated 
experience  and  training  that  has  enabled  them  to  develop  expertise.  When  operating  in  an 
environment  that  demands  immediate  action,  use  of  the  more  analytic  rational  choice 
strategy  would  require  too  much  time  to  be  effective.  Because  many  events  are  so 
dynamic,  well  thought-out  decision-making  strategies  may  not  be  applicable  when  it 
comes  time  to  institute  them. 

B.  RESEARCH  QUESTION 

The  research  question  investigated  for  this  master’s  thesis  is:  How  to  improve  first 
responder  problem-solving  and  decision-making  skills  in  today’s  new  threat 
environment? 


9  Klein,  Sources,  260. 
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C.  LITERATURE  REVIEW 

The  literature  discussed  in  this  chapter  concerning  first  responder  problem  solving 
and  decision  making  is  divided  into  three  categories:  the  asymmetrical  environment, 
decision-making  strategies,  and  scenario-based  training. 

First,  is  the  asymmetric  environment  that  first  responders  are  currently 
experiencing,  including  both  natural  and  man-made  disasters.  Experts  in  the  field  of 
terrorism  and  disaster  preparedness  (Hoffman,  Falkenrath,  and  Linstrom)  provide 
commentary  on  the  current  state  of  readiness  and  perceived  threat.  Discussions  regarding 
first  responder  levels  of  preparedness,  along  with  recommendations  designed  to  enhance 
individual  awareness  levels  regarding  terrorism  (McKinsey  Report)  and  natural  disaster 
readiness  (Federal  Response  Plan  to  Hurricane  Katrina),  underscore  the  decision-making 
process. 

Second,  are  decision-making  strategies  and  theoretical  frameworks,  especially  as 
they  pertain  to  expert  versus  novice  decision  making  and  to  rational  choice  versus 
recognition-primed  decision  strategies.  How  applicable  are  the  fire/emergency  decision¬ 
making  strategies  at  a  hurricane  or  radiological  attack?  How  does  an  expert  prioritize 
decision  making?  Klein,  Duggan,  and  Endsley  weave  intuition,  rational  choice,  and 
situation  awareness  into  a  montage  of  decision-making  analogues. 

Third  is  scenario-based  training  in  the  first  responder  decision-making  equation. 
Scenario-based  training  can  provide  effective  decision-making  training  without  the 
expense  and  risk  of  full-scale  types  of  exercises,  and  can  be  self-paced  to  accommodate 
various  levels  of  an  individual’s  level  of  expertise  and  competence.  Scenario-based 
training  helps  practitioners  connect  theory  with  real-world  applications/situations.  A 
scenario-based  experiment  (paper  and  pencil)  comparing  first  responder  experts  and 
novices  was  the  focus  of  this  thesis  research. 

1.  Asymmetric  Threats  (Terrorism/Natural  Disaster) 

In  his  paper,  Problems  of  Preparedness:  U.S.  Readiness  for  a  Domestic  Terrorist 
Attack,  Richard  A.  Falkenrath,  the  Deputy  Commissioner  of  Counterterrorism  in  the  New 
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York  Police  Department,  offers  a  suggestion  to  reduce  the  uncertainties  of  response. 
Citing  chemical  and  biological  weapons  as  examples  of  asymmetric  threats  (uncertainty), 
Falkenrath  believes  that  first  responders  need  to  develop  a  deeper  understanding  of  these 
weapons  (i.e.,  the  agent’s  properties  of  dissemination  and  interaction  with  the 
environment).  Additional  considerations  include  the  effects  of  an  asymmetric  attack  upon 
the  civilian  population,  and  how  the  civilian  reaction  will  affect  response  plans.  Will  they 
become  part  of  the  problem?  Falkenrath  suggests  that  the  public’s  reaction  to  a  weapon  of 
mass  destruction  (WMD)  incident  needs  to  be  integrated  into  the  response  plan.10  For 
example,  a  radiological  “dirty  bomb”  attack  may  require  a  mass/gross  (copious  amounts 
of  water)  or  technical  (individual)  decontamination  of  those  exposed  before  medical 
triage,  treatment,  and  transport  operations  can  begin.  The  public  will  view 
decontamination  as  necessary  or  superfluous  (depending  on  the  degree  of  contamination), 
especially  when  medical  treatment  is  vital.  How  do  first  responders  corral  possibly 
contaminated  individuals  so  they  can  be  assessed  before  they  drift  from  the  scene? 
Whereas  gross  decontamination  is  designed  to  decontaminate  large  numbers  of  people  by 
passing  them  through  a  water  shower/curtain  often  provided  by  a  hose  line,  this 
procedure  might  not  be  practical  or  well  received  in  colder  temperatures.  How  much 
experience  do  first  responders  possess  regarding  the  public’s  compliance/cooperation 
with  these  essential  procedures? 

In  his  article,  “Weapons  of  mass  destruction  a  threat  from  desperate  al-Qaeda,  ” 
Tom  Allard  reports  that,  according  to  Bruce  Hoffman,  an  expert  in  the  study  of  terrorism- 
suicide  bombers,  the  possibility  of  a  terrorist  attack  using  chemical,  biological  and 
radiological  weapons  is  growing.  These  weapons  have  not  only  a  corrosive  (physical) 
effect  but  also  a  psychological  one.  Hoffman  believes  that  the  most  likely  unconventional 
attack  would  come  from  a  radiological  or  “dirty  bomb.”  “In  a  city  like  New  York,  the 


10  Richard  A.  Falkenrath,  “Problem  of  Preparedness:  U.S.  Readiness  for  Domestic  Terrorists  Attacks,” 
International  Security,  4. (MIT  Press:  Spring  2001):  13. 

http://www.uky.edu/RGS7Patterson/desch/Readings/02-13/02-13_falkenrath_l  .html.  [Accessed  May  23, 
2007]. 
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economic  impact  would  be  immense.”11  This  type  of  device  causes  more  panic  than 
damage,  while  undennining  public  confidence  in  the  government.  Hoffman’s  concerns 
regarding  public  reaction  reinforce  Falkenrath’s  suggestions  to  incorporate  public 
reaction  into  the  response  plan.  Hoffman  further  cites  the  recent  chlorine  bomb 
explosions  in  Iraq  as  another  “unconventional”  weapon  that  caused  minimal  property 
damage  while  resulting  in  panic:  “There  are  an  endless  number  of  truck  bombs  that  have 
killed  in  the  hundreds  but  look  at  the  reaction  of  Iraqis  to  chlorine  bombs.  They  are 
absolutely  panicked.  ”  Hoffman  feels  that  the  terrorists  are  more  dangerous  than  before 
the  September  1 1  attacks  because  they  are  more  desperate.12 

In  Psychology  of  Terrorism,  Richard  Gist  identifies  the  inherent  resiliency  of 
today’s  first  responders  when  it  comes  to  post-traumatic  disorders  from  the  nature  and 
emotional  impact  of  their  work.  Several  reports  regarding  post-traumatic  stress  disorder 
(PTSD)  reveal  that  first  responders  are  “uncommonly  resilient.”13  This  resiliency  enables 
firefighters  to  provide  a  social  function  that  must  be  exercised  for  communities  to 
prosper.14  While  fire  organizations  are  their  communities’  first  resources,  providing  the 
nonnal  functions  related  to  hazards  and  safety,  many  fire  organizations  now  require  the 
expertise  regarding  hazardous  materials  identification  and  mitigation,  mass  casualty 
triage,  decontamination,  and  technical  rescues  from  natural  disaster  and  structural 
collapse.15  Acquisition  of  this  expertise  is  correlated  with  decision  making  in  two  distinct 
ways.  First,  the  recognition  of  first  responder  resiliency,  understanding  that  firefighters 
do  not  get  paid  for  what  they  do,  but  rather  for  what  they  might  have  to  do,  makes  it 


1  Horn  Allard,  “Weapons  of  mass  destruction  a  threat  from  desperate  al-Qaeda,”  Sydney  Morning 
Herald  (May  24,  2007).  http://www.smh.com.au/news/world/weapons-of-mass-destruction-a-threat-from- 
desperate-alqaeda/2007/05/23/1 179601492715.html.  [Accessed  May  5,  2007].  n.p. 

12  Ibid. 

13  Richard  Gist,  “Promoting  Resilience  and  Recovery  in  First  Responders,”  (eds)  Bruce  Bongar,  Lisa 
M.  Brown,  Larry  E.  Beutler,  James  Breckenridge,  Philip  G.  Zimbardo,  Psychology’  of  Terrorism  (New 
York:  Oxford  University  Press,  2007):  418. 

14  Ibid.,  419. 

15  Gist,  “Promoting  Resilience  and  Recovery  in  First  Responders,”  418. 
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imperative  that  decision  making  is  at  an  expert  level.  Second  is  the  recognition  that 
additional  expertise  related  to  natural  disasters  preparedness  is  in  accord  with  current 
asymmetrical  threat  planning. 

John  Linstrom  retired  at  the  rank  of  assistant  chief  after  twenty  years  of  service  in 
fire  departments  in  Texas  and  California;  presently,  he  is  an  educational  consultant  for 
the  Fire  and  Emergency  Training  Network  (FETN).  Linstrom’s  article,  “Decision  Making 
on  High  Alert  with  New  Threats,  ”  reports  that  most  responses  are  to  fires,  emergencies, 
and  medical  events  that  are  small  and  manageable.  However,  in  today’s  asymmetric 
threat  environment,  officers  must  be  alert  to  responses  that  appear  to  be  routine,  but  in 
fact  are  planned  terrorist  attacks.  Lindstrom  reports  that  80  percent  of  terrorist  attacks 
involve  the  use  of  explosives.16  According  to  Grunnar  J.  Kuepper,  the  Chief  of 
Operations  with  Emergency  and  Disaster  Management  Inc.,  Los  Angeles,  chemical, 
biological,  and  nuclear  weapons  pose  the  most  significant  challenges  to  emergency 
responders,  but  the  weapons  of  choice  creating  an  “asymmetric  threat  environment”  are 
more  often  guns  or  explosives.  17 

At  the  Madrid  train  bombings,  secondary  explosive  devices  were  placed  to  target 
first  responders.  In  a  Lessons  Learned  Infonnation  Sharing  (LLIS)  report  entitled 
Secondary  Attacks:  Failure  to  Perform  Adequate  Site  Inspection  Procedures,  the  Madrid 
bombings  highlight  concerns  about  secondary  attacks.  At  this  incident,  all  unclaimed 
personal  belongings  were  removed  to  the  police  station.  Included  in  these  belongings  was 
a  bag  containing  an  explosive  device  wired  to  a  cell  phone.  Fortunately,  the  cell  phone 
and  explosives  were  not  properly  connected,  and  the  device  failed  to  detonate.  Lessons 
learned  underline  the  inadequate  site  security  and  inspection  procedures,  while 
underscoring  the  cautions  that  need  to  be  taken  at  terrorist  sites.18  These  cautions  need  to 
be  the  stimulus  for  enhancing  first  responder  asymmetrical  decision  making  on  a  tactical 

16  John  Linstrom,  "Decision  Making  on  High  Alert  with  New  Threats,  ”  Fire  Chief  (April  2004):  29. 
[Accessed  Proquest,  January  2007]. 

17  Grunnar  J  Kuepper,  “Shooting  at  LAX  on  July  4,  2004:  Lessons  Learned,”  IAEM Bulletin  (October 
2002):  20.  http://www.LLIS.gov.  [Accessed  May  25,  2007]. 

18“Secondary  Attacks:  Failure  to  Perform  Adequate  Site  Inspection  Procedures”  (September  15, 

2004).  http://www.LLIS.gov.  [Accessed  May  25,  2007]. 
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and  strategic  level.  By  sharing  lessons  learned  from  similar  incidents,  first  responders 
operating  at  subsequent  incidents  will  be  more  attentive  to  cues  and  aware  of  their 
importance.  Via  experience,  they  will  have  a  greater  capacity  to  connect  the  past  and 
present,  and  not  miss  a  cue/pattern  that  was  overlooked  because  the 
knowledge/experience  was  missing. 

Steve  Lambakis  et  ah,  report  in  Understanding  “Asymmetric”  Threats  to  the 
United  States,  that  a  biological  weapon’s  attack  against  a  population  will  have  a  much 
different  effect  than  the  same  attack  in  a  battlefield.  Although  masks  are  an  acceptable 
means  of  protection  against  such  an  attack,  will  the  masks  and  knowledge  of  attack  be 
available?  The  United  States  does  not  have  the  monitoring  capabilities  to  detect  all  types 
of  agents.  Unknown  or  generically  altered  agents  will  not  be  in  the  monitor’s  “library.” 
Additionally,  the  Lambakis  report  claims  that  there  are  millions  of  locations  where  a 
detector  can  be  placed;  obviously  not  every  location  can  be  protected.  Attackers  know 
which  agents  have  a  vaccine  and  will  avoid  the  use  of  particular  agents.  Lastly, 
effective  treatment  depends  on  a  rapid  diagnosis.19 

A  review  of  the  McKinsey  findings  reveals  that  first  responder  experience  and 
prior  training  (emergency  response  drills)  allowed  first  responders  to  think  and  act 
intuitively  at  the  Pentagon  on  September  11,  and  to  make  effective  decisions.  Conversely, 
due  to  a  lack  of  familiarity  and  experience  concerning  today’s  asymmetrical  threat 
environment,  first  responder  decision  making  at  Ground  Zero20  and  Hurricane  Katrina21 
in  the  days  following  the  event  was  not  as  effective.  Analyses  of  such  events  have 
underscored  the  need  for  improved  decision-making  skills  and  provided  tangible  proof 
that  recovery  efforts  were  adversely  affected  when  the  decision-making  and  problem¬ 
solving  tools  were  not  available. 


19  Steven  Lambakis,  James  Kiras,  Fristin  Kolet,  “Understanding  ‘Asymmetric’  Threats  to  the  United 
States,”  National  Institute  for  Public  Policy  (September  2002):  33.  http://www.nipp.org.  [Accessed  May 
23,2007], 

20  New  York  Police  Department,  “Improving  NYPD  Emergency  Preparedness  and  Response,”  (New 
York:  McKinsey  and  Company,  2002). 
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A  RAND  report  titled,  Protecting  Emergency  Responders:  Lessons  Learned  From 
Terrorist  Attacks,  reviewed  three  first  responder  issues  in  the  days  following  a  terrorist 
attack,  and  how  prior  training  may  have  assisted  decision  making.  The  successful 
establishment  of  a  perimeter  control  at  the  two  sites  (Pentagon  and  Ground  Zero)  was 
influenced  by  many  factors;  the  Pentagon  was  in  a  rural  setting  with  limited  access,  while 
Ground  Zero  was  in  an  urban  environment  with  many  access  points.  However,  the 
number  of  access  points  and  site  location  were  not  the  sole  contributing  factors 
determining  the  accomplishment  of  the  objective.  For  example,  in  reviewing  post 
operations  at  the  Pentagon,  the  RAND  report  points  out  that  “Federal,  state,  and  local 
agencies  involved  were  accustomed  to  working  with  each  other  and  had  practiced 
emergency-response  drills.  Several  firefighters  agreed:  ‘What  made  it  [effective]  was  the 
training  that  we  do  all  the  time  with  the  other  jurisdictions,  commands,  and  knowing  the 
people.’”22 

A  post-9/ 1 1  report  prepared  by  McKinsey  and  Company  for  the  NYPD  identifies 
site  security  at  Ground  Zero  in  the  days  immediately  following  September  11,  2001,  as 
inadequate.  Perimeter  security  was  not  adequately  established,  estimating  the  risk  of  a 
second  attack  was  not  made  a  priority,  and  leadership  was  unclear  on  how  to  obtain 
additional  resources.23  According  to  Falkenrath,  “While  preparedness  plans  should  be 
addressed  in  exercises  and  simulations,  no  exercise  can  substitute  for  real  world 
experiences,  and  because  life  safety  and  normal  life  activities  cannot  be  sacrificed, 
exercises  will  always  contain  a  certain  element  of  uncertainty  until  played  out  in  a  real 
life  experience.  The  question  that  remains  unclear  is  how  well  the  training  will  hold  up  at 
a  real  world  incident.”24 

Communication  of  risk  and  access  to  vital  information  in  an  asymmetrical  threat 
environment  is  imperative.  First  responders  need  to  make  decisions  predicated  on 

22  James  T.  Bartis,  Irene  Brahmakulam,  Ari  Houser,  Brian  A.  Jackson,  D.  J.  Peterson,  Jerry  Sollinger  , 
Tom  LaTourrette,  “Protecting  Emergency  Responders:  Lessons  Learned  From  Terrorist  Attacks.”  Rand 
and  Science  and  Technology’  Policy  Institute  (Arlington,  Virginia,  2002):  48. 
http://www.rand.org/pubs/conf_proceedings/CF176/index.html.  [Accessed  May  2,  2007]. 

23  New  York  Police  Department,  “Improving  NYPD  Emergency  Preparedness  and  Response.” 

24  Falkenrath,  “Problem  of  Preparedness,”  13. 
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dynamic,  changing  information  provided  by  numerous  agencies  —  information  and 
situations  they  may  not  have  confronted  prior  to  9/11.  According  to  Bartis  et  ah,  a 
commander’s  decision-making  concerns  included  the  following:  “The  desire  to  have 
accurate  infonnation  was  of  particular  concern  to  commanders,  who  were  making 
decisions  about  what  level  of  risk  their  personnel  would  face.  ‘You’ve  got  to  understand 
what  you’re  dealing  with,’  said  a  law-enforcement  representative.  ‘We  can  equip  our  first 
responders  to  a  certain  level  .  .  .  but  as  managers,  we  have  to  understand  what  risks  they 
are  capable  of  dealing  with  or  what  hazards  [are  out  there].”’25  Decisions  will  be 
predicated  on  the  situation  and  circumstances. 

For  example,  are  we  as  managers/leaders  prepared  to  commit  personnel  to  an 
environment  that  has  not  been  properly  evaluated,  even  though  life  and  property  may  be 
at  risk?  If  no  one  is  exiting  a  subway  station  following  reports  of  an  explosion  are  we 
prepared  to  commit  personnel  to  that  same  area,  knowing  that  they  (first  responders)  may 
become  part  of  the  problem?  If  we  do  not  make  initial  decisions  that  protect  first 
responders,  the  first  responders  will  not  be  available  to  assist  those  in  trouble. 

Consider  the  following  analogue: 

One  recommendation  that  came  out  of  the  McKinsey  and  Company  report 
prepared  for  the  FDNY  concerning  fire  department  operations  at  ground  zero  was  the 
expansion  of  procedures  for  exchanging  operational  information  with  other  agencies.26  In 
the  concluding  remarks  section  of  the  RAND  document,  commanders  discussed  the 
difficulties  with  managing  a  terrorist  attack  site:  “Given  the  understandable  difficulty  of 
making  such  decisions  in  the  midst  of  a  response  effort,  site  commanders  could  greatly 
benefit  from  guidelines  developed  in  advance  of  an  incident.”27 


25  Bartis  et  al.,  “Protecting  Emergency  Responders,”  40. 

26  Fire  Department  of  New  York,  “Increasing  FDNY's  Preparedness”  /New  York:  McKinsey  and 
Company,  2002):  57. 

27  Bartis  et  al.,  “Protecting  Emergency  Responders,”  80. 
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At  the  most  fundamental  level,  part  of  the  explanation  for  why  the 
response  to  Katrina  did  not  go  as  planned  is  that  key  decision  makers  at  all 
levels  simply  were  not  familiar  with  the  plans.28 

The  Federal  Response  Plan  to  Hurricane  Katrina:  Lessons  Learned  states  that 
many  key  decision  makers  were  not  familiar  with  the  National  Response  Plan  (NRP), 
which  led  to  ineffective  coordination  between  federal,  state,  and  city  agencies.  Local 
governments  (first  responders)  were  responsible  for  developing  a  plan  that  could  be 
integrated  into  the  NRP.  Many  of  these  plans,  however,  were  still  in  the  development 
stage  or  non-existent.  The  result  was  an  operation  void  of  SOPs,  operational  guidelines, 
and  chain  of  command  protocol.29  The  Federal  Response  Plan  reports  that  in  Louisiana, 
significant  delays  slowed  down  the  delivery  of  pre -positioned  medical,  public  health,  and 
pharmaceutical  assets: 

In  some  cases,  security  and  logistics  may  have  been  issues,  but  delays  in 
“on  the  ground”  decision  making  by  local  and  state  officials  resulted  in 
delays  in  the  delivery  of  assets  and  services  when  and  where  they  were 
needed.30 

2.  Decision  Making 

Research  on  the  problem-solving  and  decision-making  strategies  used  by  today’s 
first  responders  indicates  that  many  decisions  requiring  immediate  action  are  often  made 
intuitively,  utilizing  a  “gut  like”  decision-making  process.31  Gary  Klein  reports  that  first 
responder  decision-making  strategies  at  fires  and  emergencies  involve  experience  and 
intuitive  “gut  instinct.”32  By  definition,  one  would  believe  experience  and  intuition  to  be 
in  conflict  regarding  decision-making  styles.  The  fonner  is  built  on  past  events  tried  and 
true,  while  the  latter  is  a  “shooting  from  the  hip  response.”  In  reality,  however,  they 


28  Federal  Response  Plan  to  Hurricane  Katrina:  Lessons  Learned,  February  2006. 
http://www.whitehouse.gov/reports/katrina-lessons-learned.pdf.  [Accessed  May  2,  2007].  58. 

29  Ibid.,  202. 

30  Federal  Response  Plan  to  Hurricane  Katrina:  Lessons  Learned,  202. 

31  Klein,  Sources,  33. 

32  Ibid. 
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complement  each  other.  Experience  mentally  warehouses  information  and  intuitions  pull 
the  knowledge  from  the  shelves. 33 

In  Sources  of  Power:  How  People  Make  Decisions,  Klein  describes  the  decision¬ 
making  and  problem-solving  techniques  of  people  who  make  decisions  under  stressful, 
time-pressured  circumstances.  Klein  interviewed  firefighters,  nurses,  air  traffic 
controllers,  and  military  leaders  to  detennine  why  they  decide  to  go  left  or  right,  to  run  or 
stay  put.  What  makes  a  fire  officer  decide  to  evacuate  a  perfectly  sound  building,  only  to 
have  it  collapse  minutes  later?  “Firefighters  rely  on  recognition  of  familiarity  and 
prototypicality,”34  Klein  explains,  meaning  they  often  rely  on  mental  models  to  anticipate 
what  will  happen  next.  Although  one  would  believe  that  most  decisions  made  under  life- 
and-death  circumstances  would  require  a  more  analytical  decision-making  process 
reflective  of  the  “Seven  Step  Problem-Solving  Model”  cited  earlier  in  the  chapter,  Klein 
has  discovered  that  many  emergency  decisions  are  made  to  the  contrary.  Decision  makers 
rely  on  experience;  they  use  their  knowledge  of  past  events  to  help  them  solve  present 
day  problems.  Experience  allows  them  to  think  intuitively,  in  a  blink  of  an  eye.  A 
seasoned  attending  nurse  has  seen  enough  cases  concerning  an  infant’s  ill  health  to 
diagnosis  a  problem  and  institute  a  remedy  long  before  a  blood  test  can  be  verified. 
Nurses  can  do  this  because  they  are  exposed  to  numerous  trials  with  similar  symptoms 
(patterns).  When  there  are  enough  commonalities,  the  nurses’  perceptions  and 
comprehension  of  the  current  situation  are  compared  to  their  mental  model  of  the 
situation.  By  connecting  the  past  to  the  present,  intuition  assists  in  the  decision-making 
process. 

While  Klein  discusses  numerous  strategies  decision  makers  utilize  to  think  and 
act  intuitively,  he  also  studies  the  strategies  decision  makers  use  when  experience  is  not  a 
viable  tool,  when  decision  makers  are  in  “uncharted  waters.”  Using  mental  simulation  to 
construct  plausible  outcomes  is  one  such  strategy.  Good  situation  awareness  facilitates 
the  decision  maker  in  diagnosing  the  problem;  mental  stimulation  provides  the  ability  to 


33  Duggan,  “Coup  D’oeil:  Strategic  Intuition  in  Army  Planning,”  1. 

34  Ibid. 
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rehearse  numerous  scenarios  until  one  works.35  Mental  simulation  and  the  rational  choice 
strategy  both  involve  comparing  options  and  mentally  playing  out  the  situation  to  see  if  a 
particular  course  of  action  really  works.  Mental  simulation  is  a  rapid  decision-making 
process,  while  rational  choice  strategy  is  more  deliberate.  Also,  unlike  rational  choice 
decision  making,  mental  simulation  employs  the  art  of  “satisficing,”  namely,  taking  the 
first  option  that  works,  not  necessarily  the  best. 

The  recurring  theme  in  Klein’s  Sources  of  Power  is  that  experience  is  a  major 
contributor  to  successful  decision  making  under  pressure.  However,  when  experience  is 
not  available,  as  is  often  the  case  in  an  asymmetric  threat  environment,  experts  must 
devise  alternative  methods  to  problem  solve.  Klein’s  recognition-primed  decision  model 
(RPD)  suggests  the  use  of  mental  simulation  as  one  alternative  method  when  we  have  to 
make  sense  of  different  clues.36  Mental  simulations  allow  the  decision  maker  to  mentally 
rehearse  a  series  of  decisions.  If  they  make  sense,  the  decisions  are  implemented,  if  not, 
different  actions  are  considered.  “In  order  to  build  an  effective  mental  simulation,  we 
need  to  have  good  mental  models  (fonned  from  stored  experiences)  of  how  things 
work.”37 

Crandall  et  ah,  report  that  stored  experiences  assist  the  decision  maker  in 
recognizing  patterns/matches  during  current  situations,  building  mental  models  to  be 
stored  in  memory  and  referenced  during  future  situations,  and  recognizing  similarities 
between  like  events.38  The  art  of  correctly  sizing  up  a  fire  incident  and  deciding  whether 
it  should  be  expanded  via  the  request  for  additional  units,  scaled  down  by  reducing  unit 
commitment,  or  remaining  the  status  quo  by  keeping  the  present  alann  assignment,  is  an 
example  of  combining  experience  with  critical  thinking. 

Linstrom  employs  Klein’s  recognition-primed  decision  model  when  he  discusses 
the  various  methods  a  fire  officer  uses  to  size  up  a  situation.  According  to  Linstrom,  a 

35  Klein,  Sources,  89. 

36  Ibid. 

37  Gary  A.  Klein,  The  Power  of  Intuition  (New  York:  Currency  Doubleday,  2004):  27. 

38  Beth  Crandall  et  al.,  “Training  Decision  Makers  for  the  Real  World,”  Decision  Making  In  Action: 
Models  And  Methods  (ed).  Roberta  Calderwood  et  al.  (Norwood,  New  Jersey:  Ablex  Publishing 
Corporation,  1993)  312. 
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first  responder  matches  cues/clues  from  the  current  event  with  stored  experiences  from 
past  events.  These  experiences  are  stored  as  templates  or  schemas.  The  decision  maker 
compares  cues  such  as  the  color  of  smoke  (black,  gray,  white)  with  the  cues  experienced 
in  previous  fires  to  help  assess  what  kind  of  fire  or  stage  of  fire  they  are  dealing  with. 
Once  a  pattern  is  matched,  the  response  action  that  worked  in  the  previously  experienced 
situation  will  be  recalled  and  can  be  applied.  By  making  this  comparison,  a  first 
responder  can  mentally  draw  from  similar  situations,  recalling  which  decisions  were 
effective,  and  decide  on  an  appropriate  course  of  action  for  the  current  situation.  From 
this  analysis,  a  course  of  action  is  developed.  It  may  not  be  the  best  or  only  choice,  but  it 
is  one  that  works.39 


Imagine  that  a  fire  officer  is  responding  to  an  alarm  reporting  an  explosion  inside 
a  building.  Unbeknownst  to  the  officer,  this  is  a  terrorist-related  incident.  According  to 
Linstrom,  without  prior  knowledge  of  these  facts  and  not  possessing  the 
experience/training  to  size  up  an  incident  and  detect  possible  terrorist  related  activities, 
the  officer  might  commit  his  units  to  an  interior  attack,  exposing  them  to  a  possible 
secondary  device.40 


The  fire  officer  may  find  himself  at  a  distinct  disadvantage  because  he  does  not 
have  the  experience  to  dissect  the  situation  —  nor  the  SA  that  would  help  him  create 
pattems/matches  with  previously-experienced  situations  to  develop  similarities.  This  is 
because  terrorism  is  an  asymmetric  event,  and  many  fire  officers  have  not  yet  developed 
the  expertise  to  think  and  act  decisively  in  these  novel  situations.  Instead,  they  will  rely 
on  a  more  traditional,  rational  decision-making  model.  Linstrom  calls  this  a  “no  slide  in 
the  slide  tray”  moment:  The  cues  are  foreign  to  the  officers’  experiences  and  training.41 


39  Linstrom,  “Decision  Making,”  30. 

40  Ibid. 

41Ibid„  32. 
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“Slide  tray”  slides  are  synonymous  for  the  long-term  memory  stores  (also  referred  to  as 
“schemas”  or  templates)  that  often  separate  a  novice  from  an  expert. 

3.  Situational  Awareness 

Simply  stated,  situational  awareness  (SA)  is  the  art  of  understanding  your 
surroundings  while  developing  three  distinct  levels  of  environmental  awareness,  namely 
perception  (level  1),  comprehension  (level  2),  and  projection  (level  3).  42 

For  example,  an  experienced  fire  ground  commander,  moving  through  the  SA 
levels  of  perception,  comprehension,  and  projection,  quickly  sizes  up  the  scene 
(perception),  understands  the  possibilities  and  ramifications  of  fire  spread 
(comprehension),  and  quickly  orders  a  response  that  complements  the  present  and  future 
status  of  the  fire  (projection). 

Effective  first-responder  decision  makers  create  parallels  between  SA  levels  of 
perception,  comprehension,  and  projection  and  RPD  pattem-matching/cue 
recognition/mental  model  construct.  Is  SA  part  of  the  decision-making  process?  In 
Theoretical  Underpinnings  of  Situational  Awareness:  A  Critical  Review,  Endsley  reports 
it  is  not.  Endsley  feels  that  SA  is  the  precursor  to  decision  making.  SA  provides  a 
representation  of  the  environment;  then  the  decision  maker  decides  what  to  do,  based  on 
the  information.43  According  to  Endsley,  effective  decision  making,  leading  to  effective 
performance,  often  depends  on  high  levels  of  SA.44  But,  good  SA  is  not  a  guarantee  for 
good  decisions.  For  example,  SA  could  be  correct,  yet  the  wrong  decision  is  made. 

Marvin  Cohen  tells  us  that  good  SA  helps  experts  to  solve  problems  in  familiar 
situations  by  pattern  matching  similarities.  By  first  developing  an  accurate  SA  helps 
decision  makers  make  sense  of  unfamiliar  surroundings.  Cohen  states  that  one  way  to 


42  Mica  R.  Endsley,  “Expertise  and  Situation  Awareness,”  (eds)  Neil  Charness,  K.  Anders  Ericsson, 
Paul  J.  Feltovich,  and  Robert  R.  Hoffman,  Expertise  and  Expert  Performance  (New  York:  Cambridge 
University  Press,  2006):  634. 

43  Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  4. 

44  Endsley,  “Expertise  and  Situation  Awareness,”  634. 


18 


attain  this  skill  is  to  have  decision  makers  think  more  critically  about  novel  situations.45 
Klein,  in  his  recognition-primed  decision-making  model,  sees  a  direct  linkage  between 
recognition  of  the  situation  and  actions  taken.46  Cohen  agrees  that  recognition  is 
imperative  to  the  process;  however,  there  are  other  factors  to  be  considered.  For  example, 
what  happens  if  the  SA  is  inaccurate?  Cohen  suggests  the  process  of  meta-recognition  to 
verify  and  improve  recognition  results.  Meta-recognition  probes  for  mistakes,  corrects 
what  it  finds,  and  evaluates  the  results.  Cohen  believes  that  by  studying  the  strategies  that 
experienced  decision  makers  use  when  dealing  with  uncertainty,  training  can  be 
enhanced.47  This  is  encouraging  because  it  shares  Klein’s  view  of  cognitive  modeling 
and  expert/novice  contrasts. 

4.  Expert  vs.  Novice:  Characteristics  of  Expert  Decision  Making 

When  confronting  a  naturalistic  decision-making  (NDM)  situation  characterized 
by  uncertainty,  dynamic  conditions,  ill-structured  problems,  and  time  constraints,  experts 
focus  on  comprehending  the  situation  first  and  developing  a  course  of  action  in  the 
decision-making  process  second.48  Experts  will  spend  more  time  developing  situational 
awareness  and,  once  they  recognize  the  problem  as  familiar,  they  implement  a  course  of 
action.  Conversely,  novices  spend  more  time  comparing  different  options  because  they 
do  not  have  the  stored  experiences  (mental  models)  that  would  help  them  recognize  the 
problem.49 

Richard  Adams  of  the  Florida  Institute  of  Technology  advises  the  Federal 
Aviation  Administration  (FAA)  on  aeronautical  decision  making.  Of  particular 
importance  to  Adams  and  the  FAA  are  the  decision-making  skills  of  pilots  and  how  they 

45  Marvin  S.  Cohen  and  Bryan  B.  Thompson,  “Training  Teams  to  take  Initiative:  Critical  Thinking  In 
Novel  Situations,”  Cognitive  Technologies,  Inc  (September  11,  1999):  5.  http://www.cog- 
tech.com/Publications/PubsRM.htm.  [Accessed  April  10,  2007]. 

46  Ibid.,  4. 

47  Marvin  S.  Cohen  and  Jared  T.  Freeman,  “Thinking  Naturally  About  Uncertainty,”  Cognitive 
Technologies,  Inc  (n.d.):l.  http://www.cog-tech.com/papwer/humfac/hf_1996.pdf  [Accessed  April  2007]. 

48  H.  Lauren  Pugh,  Josephine  M.  Randal,  and  Stephen  K.  Reed,  “Differences  in  Expert  and  Novice 
Situation  Awareness  in  Naturalistic  Decision  Making,”  International  Journal  of  Human-Computer  Studies 
45,  no.  5  (November  1996).  580.  [Accessed  Pro  Quest  August  2007]. 

49  Ibid.,  582. 
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overcome  adversity  in  everyday  flights.  Adams’  identification  of  seven  distinguishable 
characteristics  of  expert  decision  making  is  gennane  to  many  fields  of  expertise  (e.g., 
pilots,  first  responders,  nurses):50 

1.  Experience  and  training  develop  an  expert’s  knowledge  base.  This 
knowledge  base  allows  the  expert  to  perceive  large  meaningful 
patterns.  The  experts  see  in  “chunks”  rather  than  individual  snippets  of 
information.  Because  pattern  recognition  occurs  so  rapidly,  it  is 
sometimes  described  as  being  intuitive. 

2.  Because  an  expert  can  construct  associations  between  the  past  and 
present,  short-term  working  memory  is  available  to  address  the 
problem  at  hand  and  the  expert  is  not  pre-occupied  attempting  to  make 
sense  of  existing  circumstances. 

3.  Expert  knowledge  is  procedural  and  goal  oriented.  Experts  know  SOPs 
and  understand  situations  where  procedures  are  useful. 

4.  Experts  solve  problems  faster  because  experts  are  (1)  faster  at  skill- 
based  tasks;  (2)  more  working  memory  capacity  is  available;  and  (3) 
employ  intuitive  decision-making  strategies  (arrive  at  solutions  faster 
without  extensive  search). 

5.  Experts  excel  at  both  routine  and  adaptive  decision-making 
opportunities.  They  can  lean  on  knowledge  or  improvisation  when 
confronting  everyday  or  novel  decision-making  situations 

6.  Experts  have  the  capability  to  revise  procedures  while  validating 
situational  interpretations  through  the  process  of  mental  simulation. 

7.  Because  experts  possess  greater  domain-specific  knowledge,  they 
spend  more  time  analyzing  and  predicting  problem  complexity 
predicated  on  the  underlying  principles  of  a  problem.  This  reflection 
results  in  better  management  regarding  the  allocation  of  time  to 
problem  solving.  51 


Good  situational  awareness  and  comprehensive  mental  models  are  two  reasons 
experts  are  better  decision  makers.  First,  when  advancing  through  the  various  levels  of 


50  Dick  McKinney,  American  Airlines,  Retired.  Decatur,  Texas,  “On  Improving  Tactical  Decision 
Making.”  3.  http ://www.crm- 

devel.org/resources/paper/On%20Improving%20Tactical%20Decision%20Making%20McKinneD.pdf. 
[Accessed  November  22,  2007]. 

51  Ibid.,  3. 
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SA,  experts  expend  more  time  making  sense  out  of  uncertain  environments.  While  a 
novice  will  become  overwhelmed  in  the  complexities  of  situational  comprehension  (SA  - 
Level  2),  the  expert  will  be  selective  regarding  what  is  and  is  not  relevant  to  the  current 
situation.  This  selectivity  translates  into  limited  choice  selection  regarding  SA  -Level  3 
projection.  The  expert  has  less  to  consider  because  they  are  able  to  eliminate  non- 
essential  cues,  they  are  therefore  more  likely  to  make  a  suitable  choice  that  solves  the 
problem  at  hand.  Conversely,  the  novice  in  the  same  situation  finds  himself  or  herself 
tangled  in  a  decision-making  quagmire  attempting  to  select  the  best  course  of  action  from 
a  diverse  menu  of  self-created  options.  Second,  exposure  to  numerous  decision-making 
situations  has  allowed  the  expert  to  develop  effective  mental  models.  As  experience 
grows,  the  expert’s  mental  models  became  more  complex  and  comprehensive. 

According  to  Crandall  et  ah,  an  expert  knows  how  to  use  his/her  domain 
knowledge,  while  the  novice  relies  on  general  knowledge.52  Experts  have  numerous 
advantages;  they  see  incidents  in  tenns  of  patterns,  they  plan  and  they  do  not  need  to 
consider  numerous  alternatives. 53An  expert  takes  the  first  option  that  works;  a  novice  will 
consider  many  options.  Experts  frame  the  problem  looking  for  similarities  and  pattern 
matches;  a  novice  will  allow  the  complexities  of  an  incident  to  confuse  what  is  relevant 
and  what  can  be  discarded.  The  good  news  is  that  as  experience  and  knowledge  grow, 
novices  often  turn  into  experts.54 


Table  1.  Comparison  of  Expert  and  Novice  Decision-Making  Characteristics 


Expert 

Novice 

Spends  more  time  developing  SA 

Spends  more  time  weighing  options55 

Uses  domain  knowledge 

Relies  on  general  knowledge 

Frames  problems  by  finding  similarities 

Allows  complexities  of  incident  to  confuse 
decision-making 

Sees  in  patterns  and  does  not  consider  multiple 
options,  takes  first  option  that  works 

Considers  multiple  options56 

52  Crandall  et  al.,  “Training  Decision  Makers  for  the  Real  World,”  312. 

53  Ibid. 

54  Ibid.,  311. 

55  Pugh,  “Differences  in  Expert  and  Novice  Situation  Awareness  in  Naturalistic  Decision  Making,” 

580. 

56  Crandall  et  al.,  “Training  Decision  Makers  for  Real  World,”  312. 
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Comparing  the  proficiency  of  an  expert  and  novice  in  their  ability  to  develop  SA, 
an  analogy  can  be  made  to  driving  on  the  freeway.  In  a  paper  titled  “Anticipatory 
Thinking,”  Klein  et  ah,  discuss  the  anticipatory  thinking  that  often  overlaps  with  the 
prediction  that  occurs  in  Level  3  of  SA.  For  example,  while  an  experienced  driver  will 
scan  the  road  for  potential  problems,  using  their  peripheral  vision  to  take  in  the  “big 
picture,”  an  inexperienced  driver  will  be  more  focused,  concentrating  not  on  potential 
problems,  but  rather  on  road  issues  in  their  immediate  line  of  sight  —  namely,  keeping 
their  car  between  the  white  lines  while  ignoring  potential  trouble  spots.57  The  novice’s 
SA  is  handicapped  via  a  “tunnel  vision”  mentality  regarding  environmental  awareness. 
They  are  unable  to  process  auxiliary  infonnation  because  all  attention  is  focused  on  the 
basic  task  itself.  Because  the  experienced  driver  can  “automatic  pilot”  certain  aspects  of 
their  driving  SA  and  decision  making,  they  are  more  attentive  to  the  weak  signals  that 
may  indicate  a  future  problem,  thereby  recognizing  a  small  problem  while  it  is  small  and 
manageable. 

To  presume  that  experts  do  not  think  before  they  act  is  not  completely  accurate. 
While  their  thought  process  may  be  ongoing  and  non-reflective,  experts  do,  time 
pennitting,  ponder  their  options.  Experts,  however,  do  not  allow  themselves  to  get  caught 
up  in  the  rules  and  reasons  behind  a  decision,  because  if  the  did  they  would  be  reverting 
back  to  a  novice  decision  maker.58  Rather  they  reflect  on  the  goals  and  actions  to  achieve 
them.  This  process  is  often  referred  to  as  “deliberate  rationality.”  However,  this 
cognitive  process  should  not  be  confused  with  a  rational  choice  analysis,  but  rather  one 
that  understands  goals  and  objectives  and  how  best  to  achieve  them.  59  Experts  employ 
deliberate  rationality  with  the  intent  to  improve  one’s  intuitive  behavior  without  resorting 
to  the  more  theory-based  actions  a  novice  may  rely  on.60  The  expert  has  synthesized  SA 

57  Gary  Klein,  Chew  Lock  Pin,  and  David  Snowden,  “Anticipatory  Thinking,”  (eds)  U.  Fisher  and  K. 
Mosier,  Proceedings  of  the  Eight  International  NDM  Conference  (Pacific  Grove,  California,  June 
2007):  120. 

58  Hubert  L.  Dreyfus.  “Intuitive,  Deliberate,  and  Calculative  Models  of  Expert  Performance,”  (eds) 
Gary  Klein  and  Caroline  E.  Zsambok,  Naturalistic  Decision  Making,  Lawrence  Erlbaum  Associates, 
Mahwah,  New  Jersey,  1997):  28. 

59  Ibid.,  28. 

60  Ibid. 
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levels  1  and  2  by  considering  goals  and  objectives  (perception  and  comprehension)  and, 
when  time  permits,  using  available  information  to  make  a  level  3  SA  (projection) 
decision. 

In  Dreyfus’  chapter,  “Intuitive,  Deliberative,  and  Calculative  Models  of 
Performance,”  he  discusses  philosopher  Martin  Heidegger’s  three  kinds  of  skilled 
responses  to  a  situation.  Heidegger  says  there  are  three  methods  of  coping,  namely  ready- 
to-hand,  present- at-hand,  and  unready-to-hand.  (1)  Everyday  coping  can  be  described  as 
ready-to-hand;  we  react  without  any  thought,  (2)  present-at-hand  coping  involves  those 
circumstances  that  are  unfamiliar  and  where  experience  is  irrelevant,  and  (3)  when  we 
resort  to  unready-to-hand  we  usually  deal  with  a  situation  where  the  decision  maker  has  a 
great  deal  of  domain  experience.  They  do  not  react  intuitively,  however,  and  instead  stop 
and  think.  This  coping  should  not  be  confused  with  rational  deliberation.  Unlike  dealing 
with  the  unknown,  deliberation  takes  place  in  a  familiar  context.  Reasons  for  this  coping 
process  can  include  situations  that  are  unusual  or  those  requiring  justification.61 

Heidegger’s  analysis  is  interesting  for  several  reasons.  First,  ready-to-hand 
coping  is  a  relative  term  because  decisions  that  seem  easy  to  some  may  be  extremely 
difficult  to  others,  merely  because  of  domain  knowledge.  Ready-to-hand  is  a  good 
example  of  previously  discussed  first  responder  experts.  Second,  present-at-hand  coping 
provides  no  resident  expert.  For  example,  “space  exploration”  has  little  available 
experiential  information  so  has  little  value  to  research.  Fastly,  first  responders  operating 
in  an  asymmetrical  environment  (novice)  appear  to  be  closely  identified  with  unready-to- 
hand  decision  making.  They  have  some  experience,  but  matters  at  hand  are  somehow 
unusual. 

A  question  concerning  the  expert/novice  relationship  may  not  be  how  we  get  a 
novice  to  think  like  an  expert,  but  instead,  how  we  get  a  novice  to  learn  like  an  expert! 
An  expert  learns  by  engaging  in  deliberate  practice,  compiling  an  extensive  experience 


61  Dreyfus,  “intuitive,  Deliberate,  and  Calculative  Models,”  27 
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bank,  obtaining  feedback,  and  gaining  new  insight  and  lessons  from  mistakes.62  Crandell 
et  al.,  describe  how  experts  fine-tune  their  knowledge  by: 

chunking  information,  recognizing  patterns  and  being  aware  of  the  critical 
information  while  disregarding  less  important  information... decisions  in  a 
natural  setting  are  determined  by  individual  experiences,  pattern 
recognition  and  creating  an  association  between  familiar  patterns  and 
actions... through  years  of  experience  and  thousands  of  trials,  experts  come 
to  see  the  world  in  patterns.63 

In  Table  2,  Klein  identifies  skills  that  can  be  taught,  along  with  methods  designed 
to  make  individuals  better  decision  makers.  Klein’s  statement  that  these  skills  are  not 
generic  suggests  that  trainers  must  tailor  the  training  to  meet  decision  maker’s  specific 
needs.64 


Table  2.  Skills  to  be  Trained  and  Methods 


Skills  Trained 

Methods  for  Training 

Situational  awareness,  pattern  matching, 
cue  learning 

Strategies  for  considering  alternative 
hypothesis  /  explanations 

Typical  cases/anomalies 

Strategies  to  detect  earlier  warning  signs 
/contingency  preparation 

Mental  models/mental  simulations 

Cognitive  modeling  and  expert  /novice 
contrasts 

Managing  uncertainty/time  pressure65 

Tactical  decision  games  /  develop 
metacognitive  skills 

5.  Rational  Choice  and  Naturalistic  Decision  Making 

An  important  prerequisite  for  effective  rational  choice  problem  solving/decision 
making  is  the  availability  of  vital  information;  time  to  compare  options,  well-defined 
goals,  and  static  conditions.  Rational  choice  decision  making  is  reliable,  it  helps  novices, 
it  is  rigorous,  and,  in  general,  can  be  applied  to  other  situations.66  The  characteristics  and 


62  Klein,  Sources,  104. 

63  Crandell  et  al.,  “Training  Decision  Makers  for  the  Real  World,”  3 14. 

64  Gary  Klein,  “An  Overview  of  Naturalistic  Decision  Making  Application,”  Naturalistic  Decision 
Making,  (eds)  Gary  Klein  and  Caroline  E.  Zsambok  (Mahwah,  New  Jersey:  Lawrence  Erlbaum  Associates, 
1997),  53. 

65  Ibid.,  54. 

66  Klein,  Sources,  29. 
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urgency  of  a  decision-making  event  often  determines  the  feasibility  and  practicality  of  a 
particular  decision-making  strategy.  This  is  not  to  suggest  that  individuals  ponder 
decision-making  strategies  before  addressing  a  problem.  Quite  the  contrary,  decision 
makers  make  decisions  as  best  they  can,  unaware  of  how  they  do  it.  The  strategies  they 
use  are  a  product  of  experience,  rather  than  an  intimate  discussion  with  oneself,  arguing 
the  advantages  of  rational  choice  over  intuition!  In  Sources  of  Power,  Klein  identifies 
situations  where  a  rational  choice  strategy  may  be  the  best  choice: 

•  Is  there  a  need  for  justification;  will  actions  be  scrutinized  by  a  higher 
authority? 

•  Conflict  Resolution:  need  to  place  all  considerations  into  a  common 
format 

•  Optimization:  finding  the  best  course  of  action,  time  to  compare  (time 
is  not  a  factor) 

•  Lack  of  experience:  lack  of  portfolio  of  experiences 

•  Conditions  are  static 

•  Goals  are  defined67 


Figure  2.  Conditions  Favoring  Intuitive  and  Analytical  Approaches.68 


67  Klein,  Sources,  96. 
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Thomas  Karp,  a  Ph.D.  candidate  at  Rushmore  University,  stated,  ‘'Intuition  and 
analysis  (rational  choice)  are  essential  complementary  components  of  effective  decision 
making,”69  which  suggests  a  mutual  dependency  between  the  two  strategies.  In  Strategic 
Intuition  in  Army  Planning,  William  Duggan,  an  associate  professor  at  Columbia 
Business  School,  explores  the  idea  of  combining  the  decision-making  strategies  of 
intuition  and  rational  choice  to  form  a  more  complete  strategy. 

Duggan  addresses  the  concept  of  “intelligent  memory,”  which  is  a  union  of 
intuition  and  rational  choice  decision  making.  He  makes  an  interesting  analogy  between 
the  human  mind  and  a  warehouse:  we  stock  our  warehouse  shelves  (human  brain)  with 
information  obtained  from  knowledge  and  experience,  storing  what  we  view  as 
important;  this  is  analysis  (rational).  According  to  Duggan,  when  confronting  a  decision¬ 
making  opportunity,  intuition  utilizes  the  stock  from  our  shelves:70  “The  divide  between 
analysis  (rational)  and  intuition  reflects  an  outmoded  view  of  the  human  mind  that 
science  no  longer  supports. .  .In  the  new  view,  analysis,  and  intuition  are  so  intertwined 
that  it  is  impossible  to  separate  the  two.  Some  scientists  call  this  ‘intelligent  memory,’ 
where  analysis  puts  elements  into  your  brain  and  intuition  pulls  them  out  and  combines 
them  into  action.”71 

In  Thinking  Naturally  About  Uncertainty ,  Marvin  Cohen  asks  the  question,  “Has 
natural  decision  making  matured  enough  to  face  uncertainty  head  on?”72  Cohen  proposes 
several  “what  if’:  scenarios.  For  example,  when  patterns  and  similarities  do  not  exist, 
can  the  decision  maker  use  mental  simulations  to  rehearse  possible  outcomes?  How  do 
you  mentally  rehearse  a  course  of  action  when  you  have  no  prior  operational  experience? 
Possibly,  using  story  building  to  help  organize  events  while  relating  a  real-life  experience 
regarding  a  similar  event  to  the  problem  at  hand  may  be  one  option. 

68  Gary  Klein,  The  Power  of  Intuition  (New  York:  Currency  Doubleday,  2004):  67. 

69Thomas  Karp,  “Intuitive  Decision  Making  in  Strategy  Management,”  Research  Paper-Rushmore 
University  (2002):  11.  http://tom.karp.as/idm.pdf  [Accessed  January  3,  2007]. 

70  Columbia  Ideas  at  Work,  “Strategic  Intuition:  The  Keys  to  Innovation,”  Columbia  Business  School 
(Summer  2006):  http://www.gsb.columbia.edu/ideas.  [Accessed  January  2006]. 

71  Duggan,  “Coup  D’oeil:  Strategic  Intuition  in  Army  Planning,”  V. 

72  Ibid.,  2. 
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6.  Computer-Based  Simulation  Technologies 


The  military  conducts  war  games  designed  to  train  commanders,  soldiers,  naval 
battle  groups,  and  wings  of  an  aircraft.  In  his  article,  The  Application  of  Existing 
Simulation  Systems  To  Emergency  Homeland  Security  Training  Needs,  Smith  suggests 
modifying  war  games  by  replacing  the  military  with  the  fire  service.  Smith  says  that  a  fire 
chief  could  replace  a  military  commander,  firefighters  replace  soldiers,  and  fire  trucks 
replace  naval  battle  groups.  The  new  enemy  would  be  a  fire,  chemical  release,  or  a 
building  collapse.  Fire  ground  commanders  would  be  required  to  solve  problems  and 
make  decisions  based  on  the  map  display  interface  of  a  building  instead  of  a  battlefield.73 
Today’s  technology  makes  this  idea  a  possibility  while  providing  first  responder  training 
with  a  “tool”  that  has  already  been  field-tested.  First  responder  communities,  much  like  a 
military  unit,  are  often  required  to  problem  solve  and  make  choices  under  stressful 
conditions.  An  asymmetrical  threat  scenario  designed  to  evaluate  current  decision¬ 
making  capabilities  can  be  developed  for  this  simulation  system. 

Douglas  Page  is  a  science  and  technology  writer.  In  his  Fire  Chief  article, 
“Simulator  Goes  to  Head  of  Class,”  Page  asks  the  question,  “How  do  you  prepare  for 
your  first  blaze/fire  situation  that  you  have  never  been  in  before?”74  Page  reports  that 
current  virtual  reality  fire  simulators  are  being  retrofitted  to  demonstrate  how  life- 
threatening  fire  conditions  can  develop.  In  his  article,  “Make  the  right  call,”  Charles 
Burkell,  the  program  chair  for  executive  development  at  the  National  Fire  Academy, 
suggests  the  following  concerning  simulation  training: 

Fire  service  organizations  need  to  include  time  pressure,  shifting 
conditions  and  information  gaps  in  training  exercises.  Simulations  and 
vicarious  experiences  are  a  way  to  build  proficiency  in  lieu  of  actually 
having  “been  there  and  done  it.”75 


73  Roger  Smith,  “The  Application  of  Existing  Simulation  Systems  To  Emergency  Homeland  Security 
Training  Needs,”  Titan  Corporation,  Simulation  Interoperability  Workshop  (Europe  2003):  3. 
http://www.modelbenders.com/papers/03E-SIW-012.pdf.  [Accessed  June  15,  2007]. 

74  Douglas  Page,  “Simulator  Goes  to  Head  of  Class,”  Fire  Chief  (December  2003):  14.  [Accessed 
Proquest,  January  2007]. 

75  Charles  J.  Burkell,  “Make  the  right  call,”  Fire  Chief  (March  1999):  45.  [Accessed  Proquest,  January 
24,  2007]. 
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Refinements  to  training  include  tactics  tested  on  a  computer  without  risking  life  and  limb, 
simulated  smoke,  hot  air,  fire  gases,  and  wind  ventilation  ramifications. 

According  to  Page,  from  a  decision-making  perspective,  first  responders  will  be 
able  to  observe  the  implications  of  their  actions.76  The  ability  to  immediately  observe 
implications  of  actions  is  a  major  advantage  of  simulator  training,  according  to  Klein.77 
Burkell  adds  that 

There  are  no  foolproof  strategies.  A  fundamental  principle  in  the  military 
is  that  commanders  must  make  decisions  within  times  of  uncertainty, 
under  great  pressure.  It  is  just  part  of  the  business.  While  highly 
experienced  command  officers  strive  for  “zero  defects,”  the  reality  is  that 
some  level  of  error  will  exist.  Improving  one's  ability  to  make  the  right 
decisions  relies  on  supplementing  actual  experience  with  simulation-based 
training.78 

Burkell  believes  there  are  not  enough  real-world  opportunities  for  first  responders 
to  develop  effective  decision-making  skills,  especially  when  operating  in  a  world  of 
uncertainty.  However,  simulation  training  -  creating  computer-based  scenarios  and 
situations  that  compel  first  responders  to  make  choices  -  is  a  conduit  to  developing 
decision-making  skills.79 

In  her  article,  “Collective  Essentials  for  Test  Pilot  Leadership  Training,”  Roxana 
Tiron,  a  writer  for  National  Defense,  reports  that  to  test  the  decision-making  and 
communication  skills  of  military  pilots,  instructors  will  often  introduce  ad  hoc  injects  or 
“curveballs”  into  the  scenario.80  These  last-minute  inserts  are  designed  to  disturb  the 
pattern  matching/recognition  process  that  the  intuitive  decision  maker  relies  on. 


76  Page,  “Simulator  Goes  to  Head  of  Class,”  14. 

77  Crandall  et  al.,  “Training  Decision  Makers  for  the  Real  World.”  314. 

78  Burkell,  “Make  the  right  call,”  43. 

79  Ibid.,  43. 

80  Roxana  Tiron,  “Collective  Simulation  Essential  For  Pilot  Leadership  Training,”  National  Defense 
(December  2004):  http://www.nationaldefensemagazine.org/issues/2004/Dec/CollectiveSimulation.htm 
[Accessed  June  15,  2007]. 
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According  to  Tiron,  when  pilots  make  a  bad  decision  during  simulation  training,  they 
have  the  opportunity  to  go  back  and  determine  where  they  went  wrong.81 

7.  Decision  Skills  Training 

“Decision  skills  training  (DST)  addresses  domain-specific  training  designed  to 

accelerate  the  transition  toward  expertise. Rather  than  attempting  to  teach  decision¬ 
making  strategies,  DST  attempts  to  enhance  a  decision  makers  experience  bank  (mental 

models)  in  specific  domains,  thereby  increasing  their  recognition  decision-making  skills. ^3 
DST  is  based  on  a  list  of  training  strategies  that  provide  opportunities  for  practitioners  to 
learn  more  rapidity.  Namely:  (1)  engaging  in  deliberate  practice;  (2)  obtaining  feedback  that 

is  accurate;  (3)  building  mental  models;  (4)  recognizing  opportunities  to  learn. 84 

DST  is  often  carried  out  via  a  low-fidelity  simulation  exercise  (pencil  and  paper) 
referred  to  as  tactical  decision  games  (TDG).  Using  a  facilitator-led  discussion, 
participants  provide  responses  (decisions)  predicated  on  carefully  constructed  scenarios 
that  are  domain-specific.  For  example,  scenarios  are  read  aloud  to  specific  decision 
points,  and  participants  are  then  given  an  agreed  upon  time  limit  to  prioritize  decision 
making  predicated  on  an  existing  scenario.  According  to  Klein  et  al.,  “Tactical  decision 
games  add  to  the  trainees'  experience  base,  prepares  them  to  respond  under  uncertainty 
and  time  pressure,  and  requires  them  to  formulate  their  intent.”85  Real  world  dilemmas 
often  include  high  levels  of  uncertainty  and  time  pressure.  Participants  are  given  a  finite 
time  limit  regarding  game-decision  responses.86 


81  Tiron,  “Collective  Simulation  Essential  For  Pilot  Leadership  Training.” 

8-  Gary  Klein,  Karol  G.  Ross,  Jennifer  L.  Schafer,  “Professional  Judgments  and  Naturalistic  Decision 
Making,”  (eds)  Neil  Charness  et  al..  The  Cambridge  Handbook  of  Expertise  and  Expert  Performance  (New 
York:  Cambridge  University  Press:  2006):  412. 

85  Gary  Klein,  Rebecca  M.  Pliske,  Michael  J.  McCloskey,  “Decision  Skills  Training:  Facilitating 
Learning  From  Experience.”  (eds)  Gary  Klein  and  Eduardo  Salas,  Linking  Expertise  and  Naturalistic 
Decision  Making  (Mahwah,  New  Jersey:  Lawrence  Erlbaum  Associates,  2001):  40. 

84  Ibid.,  41. 

85  Ibid. 

86  Appendix  C.  Decision  Skills  Training,  C-l,  http://www.nifc.gov/safety/phaseiii_chapters/WFSAS- 
AppendixC.pdf.  [Accessed  December  10,  2007]. 
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DST  scenarios  are  designed  so  novices  and  experts  alike  experience  the 
following:  (1)  limits  of  their  current  repertoire  of  mental  models;  (2)  practice  in  cue 
recognition/pattem  matching,  goal  identification  and  checking  expectancies;  (3) 
practicing  mental  simulation  via  mental  models;  (4)  receipt  of  expert  feedback  regarding 
mental  models  and  COAs;  and  (5)  comparing  their  decisions  with  others.  Because  expert 
knowledge  is  believed  to  be  tacit  knowledge,  it  is  difficult  to  share;  how  can  you  tell 
someone  to  perform  an  unconscious  process?  There  are  no  lectures  on  how  to  make 
decisions.  Instead,  a  realistic  scenario  incorporating  recognizable  cues  and  patterns 
provide  novices  the  opportunity  to  perform  and  reflect  upon  their  decisions.  Scenario 
problems  require  both  SA  and  a  decision-making  process.  Upon  completion  of  decisions, 
a  facilitator-lead  discussion  provides  trainees  with  expert  feedback  and  a  structured 
dialogue  on  recognition  and  mental  simulation  process.  87 

Incorporating  the  use  of  tactical  decision  games,  or  a  similar  method  into  first 
responder  decision-making  training  should  receive  careful  consideration  for  the  following 
reasons: 

•  They  provide  a  good  opportunity  to  practice  the  skills  that  would  be  required  in 
the  management  of  an  emergency  situation. 

•  Participants  make  decisions  while  considering  the  consequences  of  a  selected 
course  of  action,  and  have  the  opportunity  to  compare  this  with  other  possible 
courses  of  action. 

•  Decision  makers  evaluate  the  rationale  behind  decisions,  rather  than  only  focusing 
on  the  decision  made. 

•  No  limits  are  placed  on  the  decisions  that  can  be  made.  They  are  not  script-driven. 

•  Scenario-based  exercises  allow  participants  to  explore  alternative  task  strategies, 
to  collect  an  extensive  experience  bank,  and  to  enrich  experiences. 

•  Participants  develop  a  repertoire  of  patterns  of  response;  opportunities  to  practice 
recognition-primed,  rule-based  and  knowledge-based  decision  making. 

•  Trainees  receive  immediate  feedback  from  peers  about  their  solutions  to  the 
scenarios.  88 


87  Klein,  “Professional  Judgments  and  Naturalistic  Decision  Making,”  413. 

88  Margaret  T.  Crichton,  Rhona  Flin,  William  A.  R.  Rattray,  “Training  Decision  Makers  -  Tactical 
Decision  Games,”  Journal  of  Contingencies  and  Crisis  Management  8,  no.  4,  December  2000:  211. 
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8. 


Literature  Summary 


References  in  the  literature  from  real-world  terrorism  and  natural  disasters  events 
highlight  the  need  for  greater  first  responder  operational  awareness  for  this  new 
environment.  Experts  in  the  field  of  counterterrorism  (e.g.,  Falkenrath  and  Hoffman) 
believe  we  are  entering  a  period  of  first  responder  asymmetric  warfare,  with  the  growing 
threat  of  chemical,  biological,  radiological,  nuclear  and  explosive  (CBRNE)  incidents  to 
be  a  real  possibility.  Klein  et  ah,  describe  the  decision-making  strategies  that  individuals 
may  use  when  required  to  problem  solve  under  stressful  conditions,  often  comparing  a 
novice  to  an  expert.  Duggan  takes  the  process  a  step  further  by  proposing  a  possible 
unification  of  strategies  to  create  more  effective  decision  makers.  Additional  research 
needs  to  identify  the  differences  in  decision-making  strategies  between  conventional 
(routine)  and  unconventional  (asymmetric)  environments  and  how  to  bridge  this  gap. 
Training  provides  methods  for  enhancing  first  responder  decision-making  expertise  by 
including  computer-based  simulations  and  scenario-based  instruction. 

D.  HYPOTHESIS 

Through  experience  and  training,  today’s  first  responders  have  acquired  the 
decision-making  skills  to  think  and  act  decisively  when  operating  at  fires,  emergencies 
and  other  related  events.  They  demonstrate  these  skills  at  the  numerous  real-world 
incidents  they  routinely  respond  to  and  mitigate.  However,  first  responders  may  lack 
experience  for  responding  to  asymmetrical  scenarios,  and  lack  the  expertise  to 
successfully  think  through  and  mitigate  incidents  that  are  often  novel  in  nature  and  filled 
with  uncertainty. 

Several  strategies,  both  on  the  national  and  local  level,  have  addressed  this  new 

threat.  For  example,  first  responder  protocol  is  required  to  become  compliant  with  the 

National  Incident  Management  System  (NIMS).  Included  in  this  initiative  is  the 

application  of  an  Incident  Command  System  (ICS)  designed  to  assist  an  Incident 

Commander  when  managing  significant  events.  At  a  local  level,  many  cities  are  writing 

Emergency  Response  Plans  (ERP)  for  CBRNE  responses.  The  FDNY  recently  published 

an  ERP  for  a  radiological  response.  While  these  initiatives  are  forward  thinking,  they  still 
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lack  one  key  ingredient,  which  is  experience,  and  experience  entails  exposure  to  variants 
of  the  expected  situation  with  opportunities  for  trial-and-error  learning  in  order  to  master 
the  type  of  responses  that  will  be  required. 

In  essence,  this  asymmetrical  threat  environment  places  the  expert  decision  maker 
in  situations  where  they  think  and  react  more  like  a  novice  due  to  the  lack  of  an 
experience  base  from  which  to  draw.  The  goal  of  this  thesis  is  to  develop  a  strategy  for 
developing  decision-making  expertise  for  this  asymmetrical  environment.  This  strategy 
will  be  developed  by  identifying  the  problem  (lack  of  experience)  along  with 
implementing  methods  that  have  been  successful  in  the  past  such  as  simulation  and 
scenario-based  training.  Having  participants  develop  a  course  of  action  for  responding  to 
these  asymmetric  threats  is  anticipated  to  fast-forward  the  acquisition  of  experience. 

The  literature  reveals  that  first  responders  will  not  be  afforded  the  luxury  of  time 
when  confronted  with  these  new  asymmetric  threats.  They  need  to  develop  alternative 
ways  to  acquire  the  decision-making  skills  they  are  unable  to  acquire  through  experience. 
Decision-making  strategies  of  pattern  matching,  cue  recognition,  and  mental  modeling 
assist  fire  ground  commanders  when  making  “gut-like”  intuitive  decisions.  Can  these 
same  skills  be  exploited  and  applied  to  an  asymmetrical  circumstance  such  as  a 
radiological  attack?  I  believe  they  can. 

As  previously  discussed,  experience  allows  us  to:  (1)  match  a  pattern  stored  in 
memory  based  on  a  preceding  experience  with  the  situation  the  decision  maker  is 
currently  experiencing,  (2)  see  similarities  between  present  and  past  events  in  terms  of 
actions  that  will  be  successful,  and  (3)  make  rationalizations  concerning  the  current 
situation  and  use  cues  to  help  problem  solve.  A  first  responder’s  ability  to  develop 
accurate  SA  when  confronting  a  challenging  problem  can  be  enhanced  in  numerous 
ways.  One  way  is  to  create  deliberate  practice,  making  sure  they  establish  goals,  have 
practice  in  making  the  types  of  difficult  judgments  that  the  job  requires,  and  rehearse 
decision  skills  that  need  improvement.  Another  method  is  to  expose  the  first  responder  to 
numerous  decision-making  situations  via  simulator/scenario-based  training;  the  trainer 
can  provide  direct  feedback  and  evaluation,  creating  a  correlation  between  SA  and 


32 


decisions  made,  good  or  bad.89  Skills  requiring  remedial  training  can  then  be  separated 
from  the  practice  and  exercised  separately.  Developing  strategies  for  considering 
alternative  explanations  about  what  is  really  going  on  —  not  becoming  fixated  with  your 
SA  and  being  flexible  in  your  initial  judgment  —  is  another  method  for  enhancing 
decision-making  skills. 

Today’s  emergency  responders  are  making  decisions  in  a  world  filled  with 
uncertainties,  employing  decision  strategies  that  rely  on  a  combination  of  a  naturalistic 
(intuitive)  style  that  relies  more  on  experience  and  “gut  instinct”  when  making  choices 
and  a  traditional  (rational)  decision-making  process  that  is  methodical,  regimental,  time 
consuming,  and  deliberate  in  its  choice  selection  (decision-making  steps).  Developing 
good  decision-making  skills  will  enable  a  decision  maker  to  think  critically  while 
remaining  flexible  when  managing  uncertain,  asymmetric  events.  Consider  the  following: 


You  are  a  fire  lieutenant  working  a  night  tour  in  Ladder  Co.  198.  You  are  new  to 
the  area  and  unfamiliar  with  the  types  of  buildings  in  your  response  district.  At  0330 
hours,  you  receive  a  water  flow  alarm  for  a  local  factory.  Upon  arrival,  you  observe  no 
indications  of  fire  and  the  building  is  well  sealed.  The  dispatcher  tells  you  there  is  no 
additional  information  on  the  alarm.  A  few  of  the  firefighters  inform  you  that  they  get  this 
alarm  occasionally.  There  is  a  small  discharge  of  water  coming  from  the  side  of  the 
building.  The  chief  is  not  available  for  the  alarm.  It  is  getting  cold! 


Regarding  the  lieutenant’s  actions/options  at  the  reported  water  leak  response, 
was  there  not  room  for  both  a  rational  and  intuitive  decision-making  thought  process?  If 
the  lieutenant  mentally  rehearsed  possible  reasons  for  the  water  flow  alann  and  then 
performed  a  methodical  investigation  regarding  the  source  of  the  alarm,  was  he  not 
making  decisions  by  using  characteristics  of  the  situations  (namely,  static  conditions, 
need  for  justification,  time  to  compare,  uncertainty)? 


89  Klein,  Sources,  104. 
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Is  decision-making  training  designed  to  turn  a  novice  into  an  expert  decision 
maker  a  guarantee  of  effective  decisions?  Absolutely  not!  Numerous  factors  such  as 
motivation,  ambition,  and  level  of  effort  must  be  considered.  However,  if  a  novice 
decision  maker  learns  to  act  like  an  expert  when  operating  in  an  emergency  environment, 
would  it  not  make  sense  to  assume  that  expert  decision  makers  can  learn  the  same  skills 
to  make  decisions  and  solve  problems  when  operating  at  natural  disasters  and  terrorist 
events?  Experienced  decision  makers  have  the  “skills”  and  “tools”  already  at  their 
disposal.  The  question  is  how  to  retrofit  these  decision-making  skills  to  include  an 
uncertain,  asymmetrical  threat  environment.  Cohen  addresses  this  issue  when  he  asks,  “Is 
there  a  naturalistic  way  to  handle  uncertainty?”90  By  providing  exposure  to  the  skilled 
performance  that  reflect  experts’  intuitive  thought  processes  during  everyday 
emergencies,  today’s  less  experienced  decision  makers  can  begin  acquiring  the  same 
skills  through  structured  simulation/scenario-based  training,  and  receiving  feedback  on 
their  perfonnance.  The  expert/novice  contrast  is  designed  to  persuade  novices  to  learn 
from  experts,  thereby  enhancing  their  decision-making  process  at  asymmetrical  threats 
that  are  plagued  with  uncertainty,  surprise,  and  chaos. 

E  METHODOLOGY 

In  addition  to  the  literature  review,  an  in-depth  analysis  of  first  responder 
situational  awareness,  naturalistic  decision  making,  and  decision-making  stratagems  were 
conducted.  Where  applicable,  real-world  examples  are  used  to  create  a  connection 
between  theory  and  reality.  Finally,  a  scenario-based  experiment  was  conducted. 
Experimental  and  control  groups’  responses  were  compared  against  each  other  and  with 
an  expert  group  to  detennine  the  influence/effectiveness  of  training  and  expert  feedback 
during  a  time-sensitive  exercise.  Results  from  the  experiment  were  analyzed  and 
recommendations  made. 


90  Klein,  Sources,  4. 
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F. 


SUMMARY 


The  remaining  thesis  chapters  will  discuss,  in  greater  depth,  the  decision-making 
theories/strategies  that  are  part  of  the  first  responders  decision-making  matrix,  while 
striving  to  unite  decision-making  theory  and  reality.  Real-life  scenarios  provide 
applicable  examples  designed  to  illustrate  the  relevance  of  first  responder  decision¬ 
making  training  in  today’s  asymmetrical  threat  environment.  Chapter  II  will  discuss  the 
three  levels  of  Situational  Awareness  (SA)  along  with  supporting  cognitive  mechanisms; 
Chapter  III  defines  naturalistic  decision  making  including  cue  recognition,  pattern 
matching,  mental  models,  and  mental  simulations;  Chapter  IV  will  orchestrate  and 
articulate  a  stratagem  to  transform  first  responders  into  better  decision  makers  when 
confronting  uncertainty,  while  identifying  the  characteristics  that  constitute  an  effective 
decision-making  strategy.  Throughout  the  course  these  chapters,  scenario-based  examples 
will  connect  theory  with  reality. 

Chapter  V  expands  upon  the  aforementioned  scenario-based  learning  while 
clarifying  the  experimental  design  facet  of  this  research.  This  scenario-based  experiment 
tasked  two  groups  of  first  responders  (experts  and  novices)  to  problem  solve  and  make 
decisions  regarding  several  realistic  asymmetrical-type  scenarios.  The  group  of  novices 
(experimental)  received  supplementary  information/training  (independent  variable)  while 
the  other  group  (control)  received  none. 
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II.  SITUATIONAL  AWARENESS 


Chapter  II  identifies  and  defines  the  three  levels  of  situational  awareness  (SA) 
known  as  (1)  perception  (observation  of  the  incident),  (2)  comprehension  (understanding 
what  you  observe),  and  (3)  projection  (taking  action  predicated  on  understanding). 
Additionally,  Chapter  II  explains  the  supporting  cognitive  mechanisms  that  are  part  and 
parcel  of  the  development  of  SA  while  describing  their  influence  on  an  effective 
decision-making/problem-solving  process. 

“SA  is  defined  as  the  perception  of  the  elements  in  the  environment  within  a 
volume  of  time  and  space,  the  comprehension  of  their  meaning  and  the  projection  of  their 
status  in  the  near  future.”91  Individuals  perceive  an  incident,  comprehend  their  perception 
(understand  the  relationships  between  perceived  cues  and  the  current  surrounding 
environment  and  desired  goals),  and  —  through  their  analysis  of  the  correlation  between 
perception,  comprehension,  and  desired  goals  —  project  future  events. 


Imagine  a  seasoned  fire  officer  and  a  newly  promoted  probationary  firefighter 
standing  in  front  of  a  burning  building,  trying  to  size  up  the  situation.  The  building  in 
question  is  a  four-story  wood  frame  building  in  a  row  of  similar  attached  buildings.  The 
fire  is  on  the  top  floor.  Both  individuals  perceive  the  same  set  of  circumstances  (top  floor 
fire).  However,  while  the  seasoned  officer  understands  that  the  fire  will  now  begin  to 
spread  horizontally,  the  less  experienced  firefighter  may  not.  Using  existing  cues,  both 
individuals  have  entered  level  2  SA.  One  of  them,  however,  is  at  a  distinct  disadvantage! 


Experienced  fire  commanders  begin  their  decision-making  process  by  perceiving 
the  elements  of  an  event  (fire  conditions  and  resources  present)  and  then  comparing  these 
observations  with  templates  stored  in  memory.  These  templates  are  based  on  previous 
training  and  experience.  For  example,  do  operational  goals  include  extinguishing  the  fire, 


91Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  3. 
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or  rescuing  occupants  and  protecting  exposures  while  conceding  the  fire  building? 
Integrating  multiple  pieces  of  infonnation  and  comprehending  the  relationship  between 
elements  of  the  problem  situation  and  operational  goals  is  significant  in  the  decision¬ 
making  progression. 

Continuing  with  the  earlier  scenario,  the  fire  commander  conducts  an  exterior 
evaluation  (size-up)  of  a  top  floor  fire  in  a  row  of  attached  wood-frame  buildings 
(perception).  Understanding  that  fires  have  a  tendency  to  extend  horizontally  when  they 
can  no  longer  extend  vertically  (comprehension),  the  commander  orders  the  stretching  of 
a  hose  line  to  the  top  floor  of  the  adjoining  attached  structure  to  stop  possible  extension 
of  fire  (projection).  However,  by  the  time  the  hose  line  reaches  its  intended  destination, 
the  fire  has  already  gained  a  foothold  in  the  adjoining  structure.  When  the  desired 
objectives/goals  of  fire  extinguishment  (prevent  fire  extension)  are  in  conflict  with 
existing  conditions  (fire  in  adjoining  exposure),  the  commander  makes  contingency  plans 
by  calling  for  additional  help  (projection). 

While  a  large  majority  of  SA  development  involves  the  systematic  process  of 
collecting,  understanding,  and  forecasting,  the  transition  from  one  SA  level  to  the  next 
does  not  always  follow  a  linear  pattern.  For  example,  while  progression  from  one  level  to 
the  next  may  be  indicative  of  a  novice  decision  maker  confronting  uncertainty  or  novel 
circumstances,  an  expert  with  good  schemata  may  not  need  to  process  information 
through  levels  2  and  3  SA.92  Perception  or  size-up  (level  1)  leads  to  projection  or  action 
(level  3).  Good  schemata  results  in  minimal  conscious  deliberation.  Comprehension  is 
less  deliberate  and  more  intuitive.  In  brief,  a  schema  is  a  person’s  prototypical  mental 
model  for  certain  situations;  they  recognize  the  situation  as  typical  and  have  a 
predetermined  “game  plan”  or  schema  to  implement.93  Mental  models,  stored  in  long¬ 
term  memory,  help  to  explain  the  way  the  world  works.  “Mental  models  provide  the  basis 
for  describing  causal  relations  among  events,  for  predicting  what  will  happen  as  a 


92  Endsley,  “Expertise  and  Situation  Awareness,”  639. 

93  Ibid.,  639. 


38 


consequence  of  an  event...”94  Prototypical  situations  can  be  learned  through  experience 
or  deliberate  practice.95  For  example,  deliberate  practice  in  the  form  of 
simulations/scenario-based  exercises  can  be  designed  around  patterns  of  recognition  that 
would  normally  take  years  to  experience  naturally.96  Because  the  infonnation  is  stored  in 
long-term  memory,  the  expert  recalls  the  schema  and  applies  it  to  the  current  situation.97 

Use  of  schemas  (mental  models)  is  one  type  of  decision-making  strategy  for 
managing  a  problem.  Therefore,  it  would  seem  logical  to  believe  that  the  greater  the 
number  of  opportunities  a  first  responder  has  to  utilize  stored  schemas  in  the  decision¬ 
making  process,  the  faster  and  more  intuitive  they  will  become.  In  natural  disasters  and 
terrorism-related  threat  environments  filled  with  uncertainty,  however,  first  responders 
may  not  have  stored  schemas  they  can  draw  on  to  help  them  recognize  a  situation  and 
know  how  to  respond.  The  development  of  good  SA,  as  described  in  the  proceeding 
chapter,  may  help  to  overcome  this  experience  deficit. 

A.  LEVEL  1  SA:  PERCEPTION 

Level  1  SA  is  defined  as  the  “perception  and  awareness  of  elements  in  the 
environment.”98  This  level  involves  knowledge  of  surroundings,  recognizing  familiarities 
along  with  inconsistencies,  and  sizing  up  an  incident  through  observation  and  surface 
identification.99  The  ability  to  recognize  elements  that  do  not  require  immediate 
investigation  or  cataloging  typifies  an  expert  operating  in  level  1  SA.  This  type  of 
situation  assessment  often  results  in  a  familiar  “been  there  before”  moment,  initiating  the 


94  Judith  M.  Orasanu,  “Shared  problem  models  and  flight  crew  performance.”  (eds)  N.  Johnston,  N. 
McDonald,  and  R.  Fuller,  Aviation  Psychology’  in  Practice  (Aldershot,  England:  Ashgate  Publishing  Group, 
1994).  2- 

95  Endsley,  “Expertise  and  Situation  Awareness,”  639. 

96  Beth  Crandell,  T.  Owens  Jacobs,  Barbara  Means,  Eduardo  Salas,  “Training  Decision  Makers  for  the 
Real  World,”  (eds)  Roberta  Calderwood,  Gary  A.  Klein,  Judith  Orasanu,  Caroline  E.  Zsambok,  Decision 
Making  in  Action:  Models  and  Methods  (Norwood,  New  Jersey:  Ablex,  1993),  307. 

97  Mica  R.  Endsley,  “The  Role  of  Situation  Awareness  in  Naturalistic  Decision  Making,”  (eds)  Gary 
Klein  and  Caroline  E.  Zsambok,  Naturalistic  Decision  Making  (Mahwah,  New  Jersey:  Lawrence  Erlbaum 
Associates,  1997):  270. 

98  Endsley,  “Expertise  and  Situation  Awareness,  ”  634. 

99  Ibid.,  270. 
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cognitive  process  that  allows  for  a  decision  without  proceeding  through  successive  SA 
levels.100  The  experienced  decision  maker  can  quickly  determine  that  the  situation  is 
benign  and  requires  no  action.  However,  there  is  a  down  side:  An  inaccurate  assessment 
at  this  level  increases  the  odds  of  an  incorrect  picture  fonning.  “Studies  found  that  76 
percent  of  SA  errors  in  pilots  could  be  traced  to  problems  in  perception  of  needed 
information.”101 


An  analogy  can  be  made  to  a  fire  commander’s  initial  SA  size-up  at  a  building 
collapse.  The  commander  performs  level  1  SA  of  this  incident  by  conducting  a  surface 
reconnaissance  of  the  area,  creating  a  mental  blueprint”  of  existing  surroundings.  For 
instance,  existing  voids  created  from  fallen  debris  may  be  apparent  or  perceived; 
however ,  exploration  considerations  involve  an  understanding/comprehension  of  “risk 
vs.  reward”  (placing  firefighters’  lives  in  danger  for  unknown  life  hazard)  consequences 
and  are  part  of  the  subsequent  level  of  SA. 


B.  LEVEL  2  SA:  COMPREHENSION 

Level  2  SA  involves  the  comprehension  of  surroundings,  helping  the  decision 
maker  to  recognize  the  relationship  of  elements  identified  in  the  current  situation,  and 
understand  how  level  1  SA  (perception)  affected  an  individual’s  goals  and  objectives.102 
For  example,  it  is  during  level  2  SA  that  the  fire  commander  will  detennine  the  relevance 
of  void  exploration  by  considering  defined  goals  (rescue)  in  conjunction  with 
comprehension  of  level  1  SA  acuity  (risk  vs.  reward).  At  level  2  SA,  novice  decision 
makers  may  experience  their  first  frustration  with  uncertainty,  failing  to  integrate  goals 


100  Endsley,  “Expertise  and  Situation  Awareness,  ”  639. 

101  Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  3. 

102  William  M.  Jones  and  Mica  Endsley,  United  States  Air  Force  Armstrong  Laboratory,  “Situation 
Awareness  Information  Dominance  &  Information  Warfare”  (February  1997):  14. 

http://mentalmodels.mitre.org/cog  eng/reference  documents/situation%20awareness%20information%20d 
ominance%20information%20warfare.pdf  [Accessed  November  2007]. 
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and  data  to  achieve  comprehension.103  Although  they  can  conduct  an  effective  level  1  SA 
investigation  (perception),  they  often  have  difficulty  converting  perception  into 
comprehension  to  identify  the  significance  of  events,  and  thus  are  unprepared 
to  make  good  decisions  at  the  following  level.104 

C.  LEVEL  3  SA:  PROJECTION 

Level  3  SA  involves  the  projection  of  future  events,  governed  by  the  decision 
maker’s  synthesis  of  perception  acuity  and  comprehension.  Detennination  of  situational 
elements  and  their  potential  impacts  upon  the  environment  is  imperative  at  this  level.105 
Knowledge  regarding  elements  of  the  environment,  in  conjunction  with  an  understanding 
of  the  dynamic  situation,  helps  to  form  level  3  SA. 


The  fire  commander ’s  goals/objectives  (saving  lives  /  protecting  property)  have 
been  aligned  with  his  current  understanding  of  the  environmental  elements  (unsafe 
operating  conditions).  Based  on  this  understanding,  he  believes  there  is  no  immediate  life 
hazard.  His  decision  to  delay  void  exploration  until  dangerous,  unexplored  areas  can  be 
adequately  reinforced  with  building  materials  has  been  made. 


“In  almost  every  field  I  have  studied  (aircraft,  air  traffic  control,  power  plant 
operations,  maintenance,  and  medicine),  I  have  found  that  experienced  operators  rely  on 
future  projections  heavily.  It  is  the  mark  of  a  skilled  expert.”106  Level  3  SA  is  the 
commander’s  ability  to  “see  over  the  horizon”  regarding  current  circumstances  and  their 
relationship  to  future  events.  Where  is  this  fire  going?  Do  I  have  enough  resources?  Do  I 
need  additional  alarms  according  to  my  understanding  of  the  situation?  Having  the 


103  Endsley,  “The  Role  of  Situation  Awareness,”  271. 

104Ibid. 

105  Ibid. 

106  Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  4. 


41 


knowledge  to  project  future  events  predicated  on  dynamic  conditions  is  SA  at  its  highest 
level.  “This  is  analogous  to  having  a  high  level  of  reading  comprehension  as  compared  to 
just  reading  words.”107 

SA  can  proceed  in  a  progression  of  awareness  from  one  level  to  the  next  or  occur 
so  quickly  that  it  appears  to  bypass  the  comprehension  and  projection  levels  and  straight 
to  intuitive  decision  making  utilizing  experience  and  knowledge,  along  with  predisposed 
schemata  stored  in  long-term  memory.  Being  proficient  in  perceiving  the  elements  of 
your  environment  is  not  sufficient  SA.  This  is  rudimentary,  conducted  on  a  basic  level. 
At  this  level,  a  novice  and  expert  may  be  on  a  level  playing  field.  The  decision  maker 
proceeds  from  level  1  to  level  2  SA  by  successfully  linking  perception  and 
comprehension  through  considering  situational  goals.  Levels  3  SA  captures  this  synthesis 
while  projecting  the  actions  of  the  elements  upon  the  environment.108 

SA  is  an  ongoing  dynamic  process  that  evolves  throughout  the  course  of  an  event; 
goals  are  achieved,  redefined,  or  abandoned  predicated  on  situational  size-up.  A  word  of 
caution  is  required  concerning  the  relationship  between  SA  and  decision  making.  Good 
SA  does  not  imply  good  decision  making.  An  individual  can  successfully  size-up  a 
situation  and  still  make  a  bad  choice.  Conversely,  poor  SA  can  be  formed,  yet  a  good 
decision  is  made  nonetheless.109  The  following  factors  impact  the  decision-maker’s 
ability  to  develop  good  SA  and  decision-making  strategies  in  complex  settings. 

1.  Limited  Attention  and  Working  Memory 

A  decision  maker’s  limited  attention  will  have  a  direct  bearing  on  the  amount  of 
information  they  can  absorb  and  process.  Complex  situations,  dynamic  conditions,  and 
the  need  for  multiple  tasking  are  commonplace  characteristics  of  emergency  events  and 
often  result  in  certain  pieces  of  infonnation  receiving  more  attention  than  others.  The 
more  time  and  attention  an  individual  must  devote  to  infonnation  they  do  not  understand 
or  have  sufficient  knowledge  of,  the  greater  the  possibility  of  missing  a  vital  piece  of 

107  Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  3. 

108  Endsley,  “The  Role  of  Situation  Awareness,”  271. 
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information.  According  to  a  National  Transportation  Safety  Board  (NTSB)  review  of 
aircraft  accident  reports,  limited  attention  accounted  for  3 1  percent  of  accidents  involving 
human  error. 1 10 

Limited  working  memory  capacity  can  inhibit  the  ability  of  the  decision  maker  to 
perceive  the  elements  (level  1  SA),  comprehend  the  situation  (level  2  SA),  and  construct 
future  projections.  A  novice  with  limited  working  memory  or  an  expert  confronting  a 
novel  situation  will  experience  the  same  frustrations  concerning  SA.  Lacking  previous 
experience,  new  information  and  current  knowledge  must  be  analyzed  in  working 
memory;  because  working  memory  has  a  limited  capacity  it  is  quickly  exhausted, 
resulting  in  a  decision-making  process  that  may  be  seriously  limited.  The  end  result  is  an 
exorbitant  amount  of  time  devoted  to  conducting  problem  solving  and  Level  3  SA 
projections  in  working  memory.111 

2.  Default  Values 

Mental  models  often  contain  default  values  for  decision  makers  in  the  form  of 
information  that  is  germane  to  the  particular  classification/domain.  Default  values  assist 
the  decision  process  by  providing  expected  characterizations  of  an  element  —  certain 
understandings  that  may  be  applied  to  the  situation  in  a  new  manner.  Default  information 
is  useful  when  the  decision  maker  is  dealing  in  areas  of  uncertainty  while  still  possessing 
some  background  knowledge,  allowing  the  expert  to  continue  in  situations  that  often 
stymie  the  novice.  This  is  true,  not  because  they  are  experts,  but  because  they  possess  a 
relevant  mental  model  developed  through  experience,  explaining  how  a  particular  system 
works.  Default  information  is  helpful  in  situations  where  there  is  missing  information  or 
overload.112  Consider  the  following  example. 


110  Endsley,  “The  Role  of  Situation  Awareness,”  272. 

111  Ibid. 
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When  operating  in  domains  of  uncertainty,  a  specialist  trained  in  hazardous 
material  operations  (expert)  possesses  technical  knowledge  that  is  germane  or  applicable 
to  many  hazardous  material  situations.  At  a  hazardous  material  operation,  this  officer 
knows  that  chemicals  contain  particular  characteristics  (lighter  or  heavier  than  air). 
They  may  not  be  able  to  identify  the  exact  agent  right  away;  however,  they  may  be  able  to 
tell  you  what  it  is  not.  This  information  leads  to  actions/options  that  may  have  stymied 
the  novice  or  uninformed. 


3.  Automatic  Action  Selection 

Automatic  action  selection  is  based  on  the  decision  maker’s  use  of  templates 
(scripts)  of  prototypical  situations  when  making  decisions.  When  operating  by  using  a 
template,  less  time  is  wasted  on  consideration  of  an  alternative  plan,  thus  freeing  up 
memory  space  while  allowing  the  decision  maker  to  process  larger  chunks  of  information 
and  make  rapid  decisions.113  Because  a  response  for  a  particular  situation  has  been 
successful  in  the  past,  individuals  do  not  have  to  search  their  memory  bank.  This  is  an 
example  of  Klein’s,  single-step  “recognition-primed”  decision  model  where  the  decision 
maker  recognizes  the  situation  based  on  comparing  cues  from  the  current  situation  with 
stored  templates,  and  the  appropriate  response  is  recognized.  The  decision  maker  does 
not  need  to  use  the  more  time-consuming  decision  process  that  involves  weighing  of 
options  and  evaluating  different  courses  of  action. 

While  the  automatic  action  selection  mechanism  provides  good  perfonnance 
coupled  with  a  low  level  of  attention  demand,  there  may  be  a  down  side.  Namely, 
because  the  decision  maker  is  on  “automatic  pilot,”  they  may  miss  a  novel  situation  that 
would  normally  be  recognized.  For  example,  would  a  person  driving  a  familiar  route 
suddenly  recognize  the  insertion  of  a  new  stop  sign?  Probably  not,  because  automatic 


113  Endsley,  “The  Role  of  Situation  Awareness,”  276. 


44 


action  selection  often  results  in  the  decision  maker  acting  on  cues  (stimulus),  short- 
circuiting  SA  and  decision  making,  and  choosing  a  CO  A  (response).  114 


World  Situation 


Cues 


Awareness 


Performance 
of  Action 


Stimulus 


Response 


r 


Automaticifcy 


F igure  3 .  Automaticity  (from 1 1 5) 


4.  Categorization  of  Information 

“Chunking”  data  allows  large  amounts  of  related  information  to  be  grouped  and 
stored  under  a  single  cue.  The  more  developed  mental  models  (as  those  belonging  to  the 
hazardous  materials  fire  officer),  the  more  fine-tuned  the  categorizing  can  be.  In  essence, 
if  the  decision  maker  possesses  an  extensive  range  of  mental  models,  the  amount  of 
information  about  the  environmental  information  can  be  small  because  it  may  be  that 
only  one  cue  is  needed  to  generate  response  activity.  For  example,  a  small  amount  of 
information  from  the  environment  properly  identified  in  the  perception  stage  can  trigger  a 
mental  model  recalling  similar  situations. 


While  operating  in  the  initial  stages  of  a  building  fire  on  a  lower  floor,  the  roof 
team  reports  the  roof-plumbing  vent  “hot  to  the  touch  ”  — a  small  piece  of  information.  A 
seasoned  chief  however,  knows  that  plumbing  vents  are  often  located  in  shafts  big 
enough  to  accommodate  additional  utilities.  If  fire  has  entered  the  shaft  (causing  the 


114  Endsley,  “Theoretical  Underpinnings  of  Situational  Awareness,”  15. 
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plumbing  vent  to  become  hot),  it  will  spread  vertically.  When  the  fire  reaches  the  top 
floor  via  the  shaft,  it  will  spread  horizontally.  Translation:  the  chief  needs  help! 


A  novice  may  struggle  here  because  his  or  her  mental  model  is  not  as  well 
developed,  while  an  expert,  although  possibly  operating  in  a  novel  situation,  has  enough 
critical  cues  available  to  stimulate  their  mental  model  from  past  fires  and  SOPs.116 

5.  Data-Driven  and  Goal-Driven  Processing 

Data-driven  infonnation  sets  in  motion  SA  via  a  bottom-up  decision-making 
process.  Salient  cues  from  the  environment  are  instrumental  in  guiding  the  decision 
maker  for  deciding  which  portions  of  the  environment  are  attended  to  (relevant)  and 
those  that  can  be  ignored  (irrelevant).  The  data-driven  process  is  a  bottom-up  level  1  SA 
operation.117  Conversely,  the  goal-driven  progression  is  a  top-down  decision-making 
process.  Goals  and  expectations  detennine  which  information  is  attended  to.  Synthesized 
data,  information,  and  goals  eventually  result  in  the  decision  maker  taking  action  based 
upon  current  understandings.  A  goal-driven  process  is  a  level  2  SA  operation. 
Throughout  SA  development,  there  is  a  constant  interaction  between  the  dynamic  and 
developing  chain  of  events  (bottom-up)  that  is  indicative  of  an  emergency-type  response 
and  goal-driven  protecting  of  life  and  property  that  often  dominates  the  decision  making 
(top-down).  While  the  top-down  process  will  be  predicated  on  existing  expectations  and 
goals,  environmental  cues  (bottom-up)  may  require  the  decision  maker  to  adjust  plans  or 
activate  a  new  set  of  goals  that  align  with  the  current  situation.118 


A  fire  chief  supervising  operations  at  an  advanced  fire  has  occupant  removal  and 
the  fire ’s  extinguishment  as  command  objectives/goals.  Because  the  building  is 
structurally  sound  and  still  partially  occupied,  the  decision  is  made  to  conduct  search 


Endsley,  “The  Role  of  Situation  Awareness,”  276. 

117  Ibid. 

118  Ibid.,  277. 


46 


and  rescue  operations  under  the  protection  of  an  interior  attack  hose  line.  The  chief  is 
goal-driven,  (top-down).  A  short  time  later,  the  chief  receives  reports  that  all  occupants 
were  safely  removed  from  the  building  and  the  roof  has  partially  collapsed  into  the  top 
floor.  Upon  receipt  of  new  information  (bottom-up),  the  chief  decides  to  adjust  the 
extinguishment  strategy.  The  chief’s  new  goals  now  include  a  “ risk  vs  reward” 
consideration;  interior  hose  lines  are  removed  and  units  prepare  for  an  exterior 
operation. 


6.  Expectations 

Individuals’  SA  can  be  biased  when  they  become  predisposed  to  certain  problem¬ 
solving  outcomes  based  on  standing  expectations  —  the  if  “X’  then  “Y”  mentality.  There 
is  a  tendency  to  create  a  bias  between  infonnation  they  are  in  agreement  with  while 
ignoring  infonnation  in  conflict  with  their  expectations.119  Klein  calls  this  process  a 
bending  of  the  map. . ..  believing  what  you  want  and  explaining  away  any  inconsistencies 
that  conflict  with  your  expectations.  For  example: 


Fire  extension  is  prevalent  regarding  certain  types  of  building  construction. 
While  a  row  of  attached  wood  frame  buildings  may  be  conducive  to  fire  spread  via  a 
common  cockloft  (unobstructed  open  area  between  top-floor  ceiling  and  underside  of 
roof  often  connecting  two  or  more  buildings),  a  similar  building  known  as  a  brownstone 
because  it  is  brick  construction  is  often  considered  a  one  building  fire  because  brick 
firewalls  extend  into  the  cockloft  separating  one  building  from  the  next.  Does  this  mean 
that  fire  cannot  spread  horizontally  in  a  brownstone  type  building  or  that  all  row  frame 
structures  have  common  cocklofts?  Now  consider  the  receipt  of  information  reporting 
fire  in  an  adjoining  building  when  it  contradicts  SOP  and  experience.  What  role  does 
experience  have  in  this  situation? 


119  Endsley,  “The  Role  of  Situation  Awareness,”  276. 
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When  the  data  conflicts  with  what  you  believe,  you  explain  away  the 
contradictions.120  While  it  is  true  that  individuals  will  process  infonnation  faster  if  it  is  in 
agreement  with  their  expectations  —  because  they  are  predisposed  in  their  SA  exceptions 
and  relying  more  on  working  memory  —  they  are  also  less  likely  to  notice 
miscalculations, 121not  unlike  our  driver  who  missed  the  stop  sign  because  he  was  on 
“automatic  pilot.” 


120  Klein,  Power,  148. 

121  Endsley,  “The  Role  of  Situation  Awareness,”  276. 
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III.  NATURALISTIC  DECISION  MAKING 


Decision  making  in  its  simplistic  form  occurs  as  follows:  The  individual 
experiences  a  situation  and  recognizes  relevant  cues  while  discarding  those  that  are 
irrelevant.  Recognized  cues  are  processed  into  patterns  that  are  verified  by  comparing 
them  to  mental  models.  If  the  pattern  and  the  individual’s  stored  experiences  (mental 
models)  are  similar,  and  a  “nano-second”  mental  rehearsal  of  the  action  (mental 
simulation)  reveals  a  match,  a  decision/choice  is  implemented. 

Research  indicates  that  decision  makers  in  time-compressed  situations  do  not 
typically  compare  options;122  there  is  no  comparing  of  options,  trying  to  decide  if  one 
choice  is  better  that  the  other.  If  the  decision  or  COA  selection  solves  the  immediate 
problem,  it  is  implemented.  Experience  enables  the  decision  maker  to  size-up  the 
situation,  determining  if  it  is  familiar  or  atypical,  create  a  match,  and  implement  a 
decision.  This  intuitive  process  is  completed  without  deliberation,  in  a  blink  of  the  eye. 
Many  first-responder  decisions  are  made  in  this  very  manner.  Time  is  a  rare  commodity 
at  a  fire  scene;  decisions  are  made  under  conditions  of  great  time  pressure,  dynamic 
surroundings,  and  ill-defined  goals.123 


A  fire  chief  is  seldom  the  first  unit  to  arrive  on  the  scene  of  a  developing  fire. 
Chiefs  are  often  dispatched  upon  the  receipt  of  a  preliminary  report  from  initial  units 
requesting  the  need  for  additional  resources  and  command.  When  the  chief  arrives, 
drawing  from  his  or  her  experience,  via  the  use  of  stored  mental  models  prompts 
anticipation  that  certain  actions  have  already  taken  place;  for  example,  hose  lines 
stretched,  ladders  raised,  searches  in  progress,  etc.  Data  driven  cues  include  changes  in 
fire  conditions  as  compared  to  those  earlier  detected.  Namely,  earlier  reports  regarding 
fire  extension  to  floor  above  via  the  window  is  now  receding;  gray/black  smoke  is  turning 


122  Endsley,  “The  Role  of  Situation  Awareness,”  17. 
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white;  smoke  is  observed  rising  rapidly  in  a  vertical  column  above  the  roof.  A  report 
informs  the  chief  that  all  victims  have  been  safely  removed  from  the  building. 

The  chief  develops  a  pattern  of  cues  that  is  familiar,  based  on  recognized  cues, 
while  selecting  a  response  that  is  conducive  to  an  interior  fire  attack  and  extinguishment 
efforts  based  on  mental  models  of  past  fires.  Fire  receding  in  conjunction  with  smoke 
turning  white  is  indicative  of  water  being  applied  to  the  fire;  smoke  observed  rising 
rapidly  from  the  roof  of  the  fire  building  indicates  that  a  ventilation  hole  has  been  cut  to 
relieve  upper  floors  of  smoke  and  hot  gases  while  limiting  horizontal  fire  spread.  Victim 
removal  may  no  longer  be  a  concern.  These  actions  will  make  conditions  on  the  upper 
floor  favorable  for  final  extinguishment.  Cue  recognition,  pattern  matching,  and  action 
script  were  all  part  of  the  pattern-recognition  process  behind  intuitive  decision  making. 
There  was  very  little  deliberation  involved  in  the  decision  selection. 


Chapter  III  identities  four  major  components  (cue  recognition,  pattern  matching, 
mental  models,  and  mental  simulation)  of  a  naturalistic  decision-making  process  and 
recognition-primed  decision  model  (RPD)  and  explains  their  relevance  to  a  first- 
responder’s  problem-solving  strategy  in  an  asymmetrical  threat  environment  of  terrorism 
and  natural  disasters. 

Naturalistic  Decision  Making  (NDM)  is  an  area  of  research  within  cognitive 
psychology  that  asks  how  experienced  people,  working  as  individuals  or  groups  in 
dynamic,  uncertain,  and  often  fast-paced  environments,  identity  and  assess  their 
situations,  make  decisions  and  take  actions  whose  consequences  are  meaningful  to  them 
and  to  the  larger  organization  in  which  they  operate.124 

The  RPD  model  describes  the  way  a  decision  maker  is  able  to  recognize  a 
situation,  by  identifying  typical  events  and  cues  and  comparing  pieces  of  data  with  stored 
templates,  or  schemas  of  previously  experienced  situations.  Based  on  the  response  that 
worked  during  the  previous  experience,  the  decision  maker  attempts  to  recognize  and 

124  Caroline  E.  Zsambok,  “Naturalistic  Decision  Making:  Where  Are  We  Now?”  (eds)  Gary  Klein  and 
Carline  E.  Zsambok,  Naturalistic  Decision  Making  (Mahwah,  New  Jersey:  Lawrence  Erlbaum  Associates, 
1997):  5. 
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evaluate  a  course  of  action  by  often  relying  on  their  intuition.125  Klein  defines  intuition  as 
the  way  experts  translate  experience  into  action.126  Conversely,  novice  decision  making 
or  problem  solving  situations  that  are  novel  (such  as  asymmetric  in  nature)  may  require  a 
more  structured  decision-making  strategy,  such  as  mentally  rehearsing  alternative 
options. 

Chapter  III  explores  this  question  via  a  comprehensive  description  of  cue 
recognition,  pattern  matching,  mental  models,  and  mental  simulation  while  supplying 
tangible  real  world  examples  (scenarios)  that  allow  the  reader  to  connect  theory  to  reality. 
These  scenarios  are  designed  to  enhance  the  reader’s  comprehension  of  a  NDM  strategy 
through  the  identification  of  the  recognition-primed  decision  model  (RPD)  benefits. 


Figure  4.  Recognition-Primed  Decision  Model.127 


Prior  to  developing  any  plans  or  procedures  for  improving  first  responder 
decision-making  skills  in  an  asymmetric  threat  environment,  it  is  important  to  define 
decision  making  terminology  and  relevance,  thereby  developing  an  understanding 
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regarding  where  each  component  belongs  in  the  decision-making  process.  For  example, 
what  is  the  difference  between  a  mental  model  and  a  mental  simulation?  While  mental 
simulation  is  defined  as  the  process  of  mentally  rehearsing  the  outcome  of  decisions, 
determining  if  an  option  is  workable,128  mental  models  are  the  decision  maker’s  stored 
experiences,  describing  how  things  work.129  Klein  notes  that  a  decision  maker’s  mental 
model  can  be  enhanced  via  exposure  to  numerous  opportunities,  as  in  rotating  an 
individual  through  various  jobs  and  duties  within  an  organization  until  becoming  familiar 
with  the  many  functions.130  Scenario-based  training  provides  another  method  to  build 
mental  models. 

Mental  simulations  are  tangible  activities  in  the  sense  that  they  involve  some  form 
of  scrutinizing,  measuring,  and  evaluation  every  time  a  decision  maker  weighs  an 
option.131  While  mental  simulations  create  a  picture  or  scene  within  one’s  head  and  allow 
the  scene  to  play  out  to  determine  its  feasibility,132  mental  models  are  adaptive  beliefs 
that  help  explain  and  predict  situations.133  Mental  simulations  and  mental  models  help  to 
explain  why  some  problems  can  be  solved  intuitively  while  others  need  to  be  puzzled  out. 

The  goal  for  this  chapter  is  to  develop  the  reader’s  mental  model  regarding  how 
the  decision-making  process  works  and  how  it  applies  in  everyday  first-responder 
problem  solving.  This  will  be  accomplished  by  providing  related  real-world  examples  for 
each  discussed  component.  One  of  the  main  constituents  of  an  effective  decision-making 
strategy,  regardless  of  the  domain,  is  the  acquisition  and  nurturing  of  real-life  experiences 
to  foster  the  decision  makers’  development  of  mental  models.  It  is  through  the  construct 
of  mental  models  that  the  reader  will  understand  the  synthesis  of  the  different  SA  levels 
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of  perception,  comprehension,  and  projection.134  By  providing  related  examples,  the 
learner  may  more  readily  see  the  relevance  to  his  particular  domain. 


A.  CUE  RECOGNITION 

Effective  cue  recognition  assists  the  decision-making  process  through  the 
differentiation  of  relevant  from  irrelevant  information,  immediate  from  gradual 
transitions,  and  bottom-up  data-driven  vs.  top-down  goal-driven  strategies.  For  example, 
the  decision-maker’s  ability  to  discern  cues  that  are  relevant  and  requiring  immediate 
action  —  while  considering  incident  reports  and  goals  —  all  begin  with  good  cue 
recognition.  Cues  help  the  decision  maker  to  construct  patterns  of  behavior,  building 
sense-making  frames  (data  importance)  that,  in  turn,  activate  corresponding  mental 
models. 


A  neonatal  nurse’s  intuition  interprets  an  infant’s  crying  pattern,  feeding 
irregularities,  distended  stomach,  and  lethargic  behavior  as  critical  cues  of  sepsis,  long 
before  the  ordered  blood  tests  return.  Recognizing  cues  that  are  relevant,  requiring 
immediate  action  while  influencing  previous  established  objectives/goals,  is  a  product  of 
prior  episodes  comprising  some  or  many  of  the  same  “tell-tale-signs.  ”  The  nurse  has 
“chunked’’  these  cues  into  one  universal  warning  sign.  It  takes  only  one  symptom  (cue)  to 
activate  a  response. 


A  fire  captain  evaluates  reports  of  creaking  floors,  sagging  roof,  and  leaning 
walls,  and  detennines  that  the  building  is  in  danger  of  collapse  and  needs  to  be  evacuated. 
The  captain  does  not  require  every  critical  cue  to  initiate  a  response;  he  probably  made 
the  decision  upon  the  receipt  of  any  one  report.  The  aforementioned  cues  prompted  the 
captain  to  intuitively  calculate  cue  relevancy,  action  requirement,  and  strategy. 

The  more  extensive  and  developed  the  background  and  domain-specific 
knowledge  neonatal  nurses  have  stored  in  long-tenn  memory,  the  more  efficient  they 

134  Endsley,  “Expertise  and  Situation  Awareness,”  634. 
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become  in  responding  to  emergency  situations.  How  do  the  previous  two  scenarios  draw 
parallels  between  cue  recognition,  SA,  and  decision  making?  The  nurse  and  fire  captain’s 
recognition  of  typical  from  atypical  cues,  while  comprehending  their  relevance  to  the 
current  environment  and  implications  on  future  decisions,  is  a  function  of  level  2  SA.  The 
neonatal  nurse  first  perceives  the  event  (level  1  SA)  and  subsequently  develops  a  strategy 
predicated  on  comprehension  of  cue  significance  relevant  to  goals  and  the  decision 
maker’s  expectancies  (level  2  SA).  Recognized  cues  stimulate  the  recall  of  information 
from  long-term  to  short-term  memory  while,  at  the  same  time,  validating  beliefs  or 
expectancy. 


While  responding  to  an  alarm,  a  fire  chief  is  attentive  to  progress  reports 
concerning  a  developing  situation.  The  chief  will  listen  to  radio  traffic  trying  to 
determine  if  the  team  is  “winning  or  losing’’  at  this  fire.  In  this  example,  the  radio  traffic 
provides  incident  cues.  Chiefs  sense  specific  strategic  and  tactical  operations  because 
cues  have  stimulated  prior  experience  or  mental  models;  the  chief  has  developed 
expectancies  regarding  this  inciden  t.  Upon  arrival  at  a  fire  scene,  visual  cues  such  as  fire 
volume,  smoke  color ,  and  direction  of  fire  spread  will  validate  whether  the  actions  the 
chief  had  anticipated  to  take  place  were  in  fact  performed  and  successful,  thereby 
influencing  subsequent  practical  decision  making. 


A  novice  and  expert  decision  maker  can  view  the  same  event,  detect  the  same 
cues,  and  yet  draw  different  conclusions.  Studies  indicate  the  difference  between  the  way 
an  expert  and  a  novice  utilize  cues  is  often  based  more  on  the  amount  of  weight  a 
decision  maker  assigns  to  a  cue  and  less  on  the  number  of  cues  considered.135  They 
consider  many  of  the  same  cues;  however,  when  processing  collected  infonnation, 
deciding  relevant  from  irrelevant,  the  expert  may  assign  a  greater  weight  to  one  cue  over 
the  other.  This  reinforces  the  theory  that  experts  and  novices,  when  developing  their 
perception  of  an  incident  (SA  level  1),  taking  in  all  available  infonnation,  are  equal  in 

135  James  Shanteau,  “How  Much  information  Does  An  Expert  Use?  Is  It  Relevant?”  Acta 
Psychologica,  81  (1992).  http://www.k-state.edu/psych/cws/pdf/acta_92.PDF.  [Accessed  July  16,  2007]. 
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ability.  The  decision  maker  is  taking  the  interpretation  of  cue  importance  to  a  new  level; 
not  only  are  relevant  and  irrelevant  cues  being  separated,  but  relevant  cues  are  further 
distinguished  by  level  of  importance  or  significance,  thus  helping  to  create  a  pattern  and 
solve  the  current  problem.  Consider  the  following  example. 


An  expert  and  a  novice  fire  officer ,  conducting  a  size-up  of  the  same  situation, 
will  extrapolate  three  to  four  significant  cues  to  assist  in  their  development  of  SA. 
However,  the  expert  will  assign  a  value  to  collected  cues;  in  essence,  the  expert  is 
beginning  to  reduce  the  complexity  of  the  situation  by  deciding  which  cue/s  is/are  the 
most  important.  In  contrast,  the  novice,  with  three  to  four  significant  cues  collected  in 
level  1  SA-perception,  is  often  overwhelmed  when  trying  to  make  sense  of  this 
information  in  level  2  SA-comprehension . 


Kemper  et  ah,  comparing  the  decision-making  abilities  of  experienced  and  novice 
pilots  in  time-sensitive  situations,  reveals  that  although  their  reasoning  abilities  were 
somewhat  equal,  the  experienced  pilots  excelled  because  they  possess  greater  domain- 
specific  knowledge  stored  in  their  long-tenn  memory.136  Experienced  pilots  recognized 
relevant  cues  and  developed  patterns  by  comparing  the  current  situation  with  schemas  of 
past  problem  situations.  In  this  study,  both  novice  and  expert  pilots  attempted  to  find  a 
match  in  long-term  memory,  but  only  experienced  pilots  were  able  to  match  cues  and 
recall  patterns  stored  in  long-tenn  memory  stores.  Conversely,  due  to  limited  domain 
knowledge,  novice  pilots  were  forced  to  assimilate  cues  into  short-term  memory.  Novice 
pilots  needed  to  develop  solutions  to  real-time  problems  from  cues,  while  problem 
solving  in  time -pressured,  stressful  conditions.137  Results  from  the  Kemper  et  ah,  study 
also  indicate  that  experienced  pilots  identified  more  cues  that  were  relevant  compared  to 
novice  pilots  who  reported  fewer  relevant  clues  and  more  irrelevant  ones.  Experienced 


136  Kemper  et  al.,  “Aeronautical  Decision  Making,”  184. 

137  Ibid.,  191. 
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pilots  were  also  more  attentive  to  relevant  cues.  The  best  indicator  of  cue  significance 
was  probably  the  amount  of  time  it  took  to  make  decisions  based  upon  cue  relevance.138 

Shanteau’s  study  asking  third-year  nursing  students  and  faculty  nurses  to  identify 
relevant  cues  in  a  nursing  scenario  revealed  that  the  students’  identification  of  relevant 
cues  was  20  percent  higher  than  the  faculty  nurses.  However,  upon  taking  a  course  on 
problem  solving  and  decision  making,  the  student  nurses  reduced  their  identification  of 
relevant  cues  by  17  percent.139  The  results  from  this  study  are  encouraging  in  several 
ways.  First,  these  findings  reinforce  the  critical  idea  that  the  difference  between  an  expert 
and  novice  is  not  determined  by  the  number  of  cues  they  considered,  but  by  the  relevance 
of  those  cues.  Second,  the  fact  that  the  student  nurses’  cue  recognition  skills  could  be 
improved  by  attending  a  problem-solving  and  decision-making  course  is  inspirational  to 
the  author’s  hypothesis  that  decision-making  skills  can  be  learned  by  decision  makers 
who  lack  experience. 

This  cue  filtering  process  is  perfonned  in  the  Level  2  stage  of  SA-comprehension 
and  demonstrates  a  critical  thinking  component  of  the  decision-making  progression.  The 
decision  maker  is  making  assumptions  and  decisions  based  on  intangible  information,  as 
compared  to  the  corporeal  information  that  can  be  visualized  for  authenticity.  This 
critical  thinking  will  help  the  decision  maker  to  think  more  intuitively  when  confronting 
this  same  problem  in  the  future. 

The  skills  that  help  experts  to  solve  problems  in  familiar  situations,  by  utilizing 
cue  recognition  of  similar  events  to  develop  pattern  matching,  are  the  same  skills  that  will 
help  them  make  good  decisions  in  areas  of  uncertainty  because  expert  decision  makers 
will  think  more  critically  about  novel  situations.  Cohen  suggests  that,  because  they  are 
experts,  they  have  the  ability  to  think  critically  and,  thereby,  are  more  critical  of  their 
own  assessment  —  and  less  likely  to  explain  away  situations.  Because  experts  are  so  well 
tuned  to  cues  that  foster  patterns,  they  may  very  well  be  aware,  faster  than  a  novice,  that  a 


138  Shanteau,  “How  Much  Information  Does  An  Expert  Use?”  4. 
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piece  of  information  is  missing.  140  Cohen’s  statement  regarding  experts  thinking  more 
critically  about  novel  situations  is  the  foundation  for  this  thesis.  We  know  that  experts 
may  not  have  the  knowledge/experience  regarding  asymmetric  events,  because  these 
events  are  rare.  Experts  do,  however,  know  how  to  think  critically  and,  hopefully,  that 
critical  thinking  can  influence  novice  decision  making. 

The  ultimate  goal  of  cue  collection  and  recognition  is  to  take  and  use  the 
information  to  build  patterns  of  expected  behavior  while  utilizing  mental  models  to 
confirm  beliefs.  As  vital  as  the  collection  of  cues  is  to  problem  solving,  if  cues  cannot  be 
transformed  and  arranged  into  a  coherent  pattern  of  familiarity,  they  are  of  little  use. 

For  example,  imagine  that  while  driving  along  a  freeway  there  is  a  sudden  buildup 
of  traffic,  with  little  or  no  traffic  coming  from  the  opposite  direction.  To  an  inexperienced 
driver  concentrating  on  the  minutia  details  of  the  road,  staying  attentive  to  their 
immediate  surroundings,  the  pattern  of  recognition  regarding  the  traffic  congestion  in  one 
direction  coupled  with  a  lack  of  cars  in  opposing  direction  may  never  be  comprehended. 
However,  an  experienced  driver  recognizes  this  sudden  change  in  traffic  as  a  potential 
problem  ahead  (some  type  of  road  condition/accident)  in  their  direction  of  travel  or 
possibly  in  the  opposite  direction,  while  observing  little  traffic  coming  from  the  opposite 
direction.  These  two  situational  cues  help  create  a  pattern  of  familiarity  that  is  indicative 
of  a  road  problem  ahead;  namely  heavy  traffic  in  one  direction  and  little  or  no  traffic  in 
the  opposite. 


At  this  point,  the  expert’s  ability  to  distinguish  between  relevant  and  irrelevant 
information  is  critical.  For  example,  the  novice  may  consider  incident  location  in  their 
decision-making  process,  while  the  expert  will  not.  Why?  Because  the  information 
pertaining  to  incident  location  has  no  bearing  on  the  problem  at  hand.  Regardless  of 
location,  a  decision  needs  to  be  made  involving  route  selection.  For  the  inexperienced 


Marvin  S.  Cohen  and  Bryan  B.  Thompson,  “Training  Teams  to  take  Initiative:  Critical  Thinking  In 
Novel  Situations,”  Cognitive  Technologies,  Inc.  (September  11,  1999)  :5  http://www.cog- 
tech.com/Piiblications/PubsRM.htm  [Accessed  April  10,  2007] 

57 


driver,  time  eventually  runs  out  and  they  become  part  of  the  traffic  jam,  while  the 
experienced  driver,  without  deliberation,  exits  at  the  nearest  egress  and  drives  the  service 
road  until  past  the  problem. 


B.  PATTERN  MATCHING 

“Pattern  matching,  often  performed  intuitively,  refers  to  the  ability  of  the  expert  to 
detect  typicality  and  notice  events  that  did  not  happen  and  other  anomalies  that  violate 
the  pattern.”141  Problem  solving  by  the  utilization  of  their  pattern-matching  skills,  in 
conjunction  with  intuition  and  experience,  is  often  a  first  responder’s  decision-making 
modus  operandi. 

When  dealing  with  uncertainty,  decision  makers  begin  to  identify  irregularities 
(via  cue  recognition)  based  on  their  expectations  regarding  previously  experienced  or 
known  patterns  of  behavior.  Developed  recognition  provides  patterns  for  future 
decisions.142  This  is  imperative  because  events  are  more  easily  recognized  and  re-created 
when  their  structure  makes  sense  or  a  decision  maker  has  domain-specific  knowledge. 
The  more  expertise  an  individual  has  acquired  in  a  particular  domain,  the  more  elaborate 
the  portfolio  of  experience  at  their  disposal  for  comparison  and  greater  recognition.  If 
they  are  not  experts  or  experiencing  a  novel  asymmetrical  situation,  they  may  recognize 
cues;  they  will  struggle,  however,  in  constructing  and  detecting  typical  patterns. 


For  example,  when  a  group  of  chess  masters  was  asked  to  re-create  the  positions 
of  observed  chess  pieces  on  a  board,  they  were  better  able  to  re-create  those  positions 
that  followed  a  logical  sequence  of  chess  moves  and  positions  than  those  that  were 
randomly  placed  on  the  board.143  What  this  tells  us  from  an  asymmetrical  threat 
environment  perspective  is  that  while  a  fire  chief  can  mentally  recreate  hose  line 


141  Klein,  Sources,  149. 
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positions,  fire  apparatus  placement,  and  incident  command  post  locations  while 
responding  to  a  particular  type  of  fire,  he  may  not  be  afforded  this  luxury  at  a  hurricane 
or  dirty  bomb  explosion.  In  addition,  the  individual  will  spend  an  excessive  amount  of 
time  in  working  memory,  creating  meaning  from  knowledge  of  surroundings 
—  solving  problems  but  missing  critical  cues. 


Ericsson  et  al.,  report  that  experts  are  only  experts  when  solving  problems  in  their 
own  domain.  For  example,  a  seasoned  urban  fire  commander  may  not  be  an  expert  when 
operating  at  a  rural  forest  fire.  The  risks  associated  with  experts  operating  in  novel 
situations  include  the  use  of  inappropriate  patterns  of  recognition  from  past  events.  While 
pattern  matching  is  designed  to  assist  in  the  decision-making  process,  the  decision  maker 
may  ignore  inconsistencies  to  make  the  pattern  fit.144  This  process  was  cited  earlier  as 
Klein’s  “bending  of  the  map.”  The  decision  maker  explains  away  inconsistencies  to 
justify  an  initial  pattern  match  with  what  has  worked  in  the  past.  The  rationalization  of 
anomalies  and  inconsistent  patterns  in  the  decision-making  process  only  compounds  the 
situation,  often  reaching  a  point  of  no  return.145  Revisiting  the  neonatal  nurse’s  decision¬ 
making  scenario,  consider  the  following  regarding  pattern  matching,  intuition,  and 
experience. 


The  critical  role  of  recognition  in  decision  making  came  into  sharper  focus  when 
Beth  Crandall,  51,  vice  president  of  research  operations  at  Klein  Associates,  got  a 
contract  from  the  National  Institutes  of  Health  to  study  how  intensive-care  nurses  make 
decisions.  In  1989,  she  interviewed  nineteen  nurses  who  worked  in  the  neonatal  ward  of 
Miami  Valley  Hospital  in  Dayton,  Ohio.  The  nurses  cared  for  newborns  in  distress  — 
some  post  mature,  some  premature.  When  premature  babies  develop  a  septic  condition  or 
an  infection,  it  can  rapidly  spread  throughout  their  bodies  and  kill  them.  Detecting  sepsis 
quickly  is  critical.  Crandall  heard  dozens  of  stories  from  nurses  who  would  glance  at  an 
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infant,  instantly  recognize  that  the  baby  was  succumbing  to  an  infection,  and  take 
emergency  action  to  save  the  baby's  life.  How  did  they  know  whether  to  act?  Almost 
always,  Crandall  got  the  same  answer:  "You  just  know."146 


Crandall  asked  the  nurses  to  recall  the  details  that  lead  them  to  believe  the  infant 
was  suffering  from  sepsis.  Complexion  fade  from  pink  to  gray,  changes  in  feeding 
patterns,  swollen  abdomen,  and  a  frequently  crying  baby  that  suddenly  becomes  listless 
and  lethargic  were  among  the  symptoms  that  helped  to  develop  a  pattern.  A  novice  nurse 
could  easily  explain  each  of  these  symptoms  when  they  occur  in  isolation;  however,  an 
experienced  nurse  chunks  symptom  together  to  form  a  pattern,  draws  upon  past  events 
(experience)  concerning  infants  diagnosed  with  sepsis  who  exhibited  not  all  but  many  of 
the  same  symptoms,  and  intuitively  creates  a  match.  When  infants  display  enough 
symptoms  to  develop  a  pattern  indicative  of  sepsis,  the  nurses  know  what  to  do.  Many 
times  the  baby  is  placed  on  antibiotics  days  before  blood  tests  return  confirming  sepsis. 
“Each  of  these  cues  is  extremely  subtle  but,  taken  together,  they  are  a  danger  signal  to  an 
experienced  nurse.”147  This  is  why  the  development  of  pattern  recognition  skills  is  so 
important  to  decision  making. 

Once  the  decision  maker  recognizes  a  pattern  as  familiar,  intuition  activates  action 
scripts.  We  intuitively  know  what  will  work  predicated  on  experience.  While  intuition 
has  been  described  as  a  “gut  feeling”,  it  may  be  better  described  here  as  a  not  conscious 
thought  process  because  the  person  has  seen/experienced  it  before  and  they  automatically 
recognize  it  and  can  act  based  on  recognition  of  the  situation.  We  sense  what  to  expect 
next  while  recalling  routines  (action  scripts)  for  responding.  The  more  patterns/action 
scripts  a  decision  maker  has,  and  the  more  expertise  they  possess,  the  easier  is  to  make 
decisions.148  Consider  the  following  examples  of  the  pattern  recognition  process  to 
intuitive  decision  making: 


746Bill  Breen  “What's  Your  Intuition,”  Fast  Company.com.  38,  August  2000,  290, 
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Fire  ground  commanders  have  different  action  scripts  for  each  situation.  One 
script  may  address  search  and  rescue  while  another  pertains  to  fire  confinement  and 
exposure  protection.  The  fire  commander  constructs  a  set  of  patterns,  generated  from 
cues,  organizing  information  particular  to  their  domain.  These  patterns  are  stored  in 
long-term  memory.149  The  decision  maker’s  experience  level,  pattern  recognition  skills, 
and  developed  associations  between  patterns  and  action  scripts  are  all  influential  factors 
concerning  a  fire  commander ’s  decision  making  in  a  pressured  environment.  150 


Using  pattern  recognition  in  critical  time  sensitive  situations,  matching  situational 
cues  and  patterns  with  automated  response  actions  appears  to  be  the  way  that  firefighters, 
nurses,  and  military  commanders  make  split  second  decisions.  Intuitive  decision  making 
is  predicated  on  effective  pattern  matching.151  However,  when  the  data  are  conflicting 
and  unreliable,  when  information  is  incomplete  or  ambiguous,  how  do  these  same 
individuals  perform  situational  assessments? 

C.  MENTAL  MODELS 

Experienced  decision  makers  have  internal  representations  of  the  system  they  are 
dealing  with.  They  refer  to  these  representations  as  mental  models.  152  “Mental  models 
are  adaptive  belief  constructs  used  to  describe,  explain,  and  predict  situations.”153 
Developed  from  years  of  practice,  a  mental  model  can  be  viewed  as  a  mental  outline  of 
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expected  events  for  a  dynamic  situation.154  Because  a  decision  maker’s  working  memory 
capacity  is  limited,  mental  models  stored  as  long-tenn  memory  representations  are 
retrieved  to  depict  and  predict  how  the  world  operates.155  They  allow  the  decision  maker 
to  use  long-term  memory  stores  to  solve  present  day  problems,  attending  to  the  dynamic 
unfolding  circumstances  of  the  event  at  hand. 

A  mental  model’s  domain  is  comprised  of  features,  functions,  and 
applicability.156  For  example,  an  individual  becomes  familiar  with  a  particular  piece  of 
equipment  (tool).  They  know  the  tool’s  features,  capabilities,  and  components.  They 
know  where  and  when  the  equipment  is  used  and  how  the  tool  will  react  with  the 
environment.  When  employing  this  piece  of  equipment,  the  decision  maker  compares  the 
current  situation  with  his  mental  model  of  the  equipment’s  capabilities  and  predicts  its 
effectiveness.  In  essence,  a  parallel  is  created,  comparing  the  past  (memory)  with  the 
present  situation.  If  there  is  a  match,  the  decision  maker  overlays  the  mental  model  on  the 
situational  model.  If  not,  he  takes  additional  steps  (options)  to  solve  the  problem. 

First  responders  working  in  similar  domains  for  extended  periods  of  time, 
responding  to  specific  types  of  events,  develop  mental  models  depicting  how  certain 
events  should  evolve,  the  way  things  work  —  seeing  inside  the  object  or  event.  A 
hazardous  material  technician  becomes  an  expert  in  the  handling  of  hazardous  material 
incidents  because  he  specializes  in  responses  of  this  type.  Technicians  are  often 
summoned  to  mitigate  hazardous  material  responses  that  are  beyond  the  technical 
capability  of  operating  first  responders.  By  repeated  exposure  to  domain-specific  events, 
technicians  develop  and  store  experiences  —  mental  models  from  previous  operations. 
They  become  experts. 
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A  fire  chief  assigned  to  a  high-rise  building  district  for  an  extended  period  of  time 
should  begin  to  develop  mental  models  to  aid  in  strategic  planning  and  decision  making 
when  responding  and  operating  at  high-rise  building  fires.  How  well  these  buildings 
resist  vertical  and  horizontal  fire  spread,  along  with  their  vulnerability  to  collapse,  will 
become  the  subject  of  numerous  after-action  reports  and  fire  critiques.  The  chief’s 
success  stories,  failures,  and  near  misses  will  be  fodder  for  mental  model  development. 
Decisions  that  worked,  fell  short,  or  were  classified  as  last  option  choices  will  be  filed 
away  for  future  retrieval.  The  chief  becomes  sensitive  to  any  information  that  may  trigger 
a  response  action.  His  mental  models  are  extensive  enough  to  filter  information,  deciding 
relevant  from  irrelevant,  identifying  which  information  needs  his  immediate  attention. 
When  confronting  incidents  of  a  similar  type,  the  fire  chief  relies  on  prior  knowledge  or 
mental  models  of  how  the  world  operates  according  to  prior  experience. 


It  would  seem  logical  to  assume  that  the  more  time  individuals  spend  in  particular 
domains,  the  more  numerous  and  extensive  their  mental  models  will  become.  In  essence, 
they  are  becoming  experts  in  their  domains.  Conversely,  Klein  suggests  the  rotation  of 
employees  through  the  different  jobs  of  a  corporation,  exposing  them  to  new  and 
challenging  situations  and  job  requirements,  thereby  enhancing  their  mental  models. 
When  required  to  work  with  the  different  entities  of  a  corporation,  each  employee  will 
have  greater  knowledge  concerning  job  characteristics.  Consider  the  following  example 
of  mental  model  building. 


In  an  attempt  to  expand  a  firefighter ’s  experience/knowledge  base  concerning  the 

wide-range  of  building  construction  and  fire  related  scenarios  a  firefighter  might 

confront,  the  Fire  Department  City  of  New  York  (FDNY)  established  a  training  program 

requiring  newly  promoted  firefighters  to  work  in  several  firehouses  and  locals 

throughout  their  first  three  years  of  service.  The  average  rotation  time  spent  in  each 

firehouse  was  approximately  one  year.  This  was  groundbreaking  considering  the  fact  that 

many  firefighters  work  their  entire  career  in  one  firehouse,  becoming  resident  experts 
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regarding  buildings  or  area  oddities  specific  to  their  response  district.  For  instance, 
firefighters  dedicating  their  entire  career  to  a  low-rise  building  district,  becoming  quite 
proficien  t  concerning  fire  operations  in  buildings  germane  to  their  district,  may  be  at  a 
distinct  disadvantage  if  they  suddenly  find  themselves  working  in  mid-town  Manhattan 
amongst  the  forty-  or  fifty-story  office  buildings.  By  rotating  through  several  firehouses 
scattered  throughout  the  city,  the  firefighters  have  the  opportunity  to  experience  fires  and 
emergencies  that  are  not  predicated  on  a  particular  domain.  The  goal  was  to  produce  a 
more  rounded  firefighter  with  varied  mental  models. 


Mental  models  assist  decision  making  in  two  distinct  ways.  First,  when  we  use 
mental  simulations  to  puzzle  out  problems,  mental  models  help  to  predict  outcomes. 
Second,  mental  models  help  the  decision  maker  to  recognize  inconsistencies  when  pattern 
matching  the  present  to  the  past,  recognizing  what  is  typical  and  what  is  out  of  the  nonn. 
Mental  models  are  a  type  of  story  building;  they  give  us  a  casual  account  of  how  things 
work  while  helping  us  make  educated  guesses  and  spot  inconsistencies.157 

Klein  et  al.,  take  the  mental  model  theory  to  the  next  level  by  identifying  and 
distinguishing  the  difference  between  comprehensive  models  and  just-in-time  (JIT)  models. 
Comprehensive  models  belong  to  individuals  who  possess  thorough  knowledge  in  a 
particular  domain.  For  example,  mechanics  who  work  primarily  on  diagnosing  and  repairing 
automobile  transmissions  would  possess  comprehensive  models  in  transmission  repair.  A 
transmission-related  problem  activates  long-term  memory  stores  that  retrieve  mental  models 
of  past  transmission  repair  work.  When  the  job  is  successfully  completed,  the  results  are 
stored  in  long-term  memory  for  future  use. 

“In  complex  and  open  systems,  a  comprehensive  mental  model  is  unrealistic. ”158 
While  the  comprehensive  mental  model  is  static,  first  responders  often  needdynamic, 
fragmentary  knowledge  to  help  make  sense  of  a  situation.  This  sense  making  is  the  prelude  to 
JIT  mental  models.  159 


157  Klein,  Power,  314. 

158  Klein  et  al.,  “A  Data-Frame,”  151. 

159  Ibid.,  130. 
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JIT  mental  models  rely  on  the  belief  that  fragmentary  knowledge  from  one 
domain  can  be  applied  to  problem  solving  in  another  domain.  When  decision  makers 
discover  themselves  troubleshooting,  working  with  limited  knowledge,  considering 
choices  without  a  complete  understanding  about  what  is  going  on,  they  often  utilize  JIT 
mental  models  as  an  additional  method  to  problem  solving.  The  concept  of  JIT  is  in  no 
way  related  to  time  constraints,  as  if  the  JIT  model  was  devised  at  the  last  possible 
second.  Instead,  it  is  a  theory  whereby  the  decision  maker,  because  there  is  no 
comprehensive  mental  model,  will  construct  a  mental  model  at  the  time  it  is  needed.160  A 
dividend  of  the  JIT  mental  model  process  may  be  the  attainment  and  storage  of  new 
knowledge  in  long-term  memory. 

There  are  domains  where  the  need  for  a  comprehensive  type  of  mental  model 
exists;  for  example,  a  cardiac  system  is  somewhat  closed  and  refined  as  science 
progresses.  However,  compare  this  to  the  fire  chief  standing  in  front  of  a  burning 
building,  developing  and  creating  multiple  JIT  models  to  make  effective  decisions. 

A  fire  officer  will  rely  on  knowledge/experiences  acquired  from  similar  situations 
and  develop  a  COA  based  on  what  has  worked  in  the  past.  For  example,  in  an  earlier 
scenario,  while  processing  reports  of  sagging  floors  within  the  building,  the  fire  captain 
recalls  a  similar  situation  from  long-term  memory  when  reports  of  sagging  floors  were 
followed  by  a  building  collapse.  Imagine  the  fire  captain  receiving  relevant  information 
regarding  individual  safety  but  minus  the  mental  model  that  provides  a  COA  or  the 
ability  to  construct  a  JIT  mental  model.  Translation:  The  fire  officer  does  not  need  to  be  a 
building  engineer  to  possess  enough  fragmentary  knowledge  regarding  indications  of 
building  collapse. 

Experts  are  making  better  on-the-spot  decisions  because  they  have  a  greater 
fragmentary  knowledge  base,  a  more  thorough  cause-and-effect  relationship  than  their 
counterpart  who  is  considered  a  novice.  Klein  et  ah,  suggests  that  we  train  people  by 
expanding  their  “repertoire  and  depth  of  casual  relationships,  rather  than  trying  to  teach 


160  Klein  et  at,  “A  Data-Frame,”  131. 
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fully  worked  out  mental  models  within  a  domain.”161  This  statement  is  encouraging  and 
reinforces  the  belief  that  SA,  decision  making,  and  critical  thinking  templates  can  be 
developed  for  today’s  asymmetrical  threat  environment.  Consider  the  example  and  how 
mental  models  contributed  to  an  intuitive  decision-making  scenario. 


It  is  a  simple  house  fire  in  a  one-story  house  in  a  residen  tial  neighborhood.  The 
fire  is  in  the  back,  in  the  kitchen  area.  The  lieutenant  leads  his  hose  crew  into  the 
building,  to  the  back,  to  spray  water  on  the  fire,  but  the  fire  just  roars  back  at  them. 
“Odd,  ”  he  thinks.  The  water  should  have  more  of  an  impact.  They  try  dousing  it  again, 
and  get  the  same  results.  They  retreat  a  few  steps  to  regroup.  Then  the  lieutenant  starts  to 
feel  as  if  something  is  not  right.  He  does  not  have  any  clues;  he  just  does  not  feel  right 
about  being  in  that  house,  so  he  orders  his  men  out  of  the  building  —  a  perfectly  standard 
building  with  nothing  out  of  the  ordinary.  As  soon  as  his  men  leave  the  building,  the  floor 
where  they  had  been  standing  collapses.  Had  they  still  been  inside,  they  would  have 
plunged  into  the  fire  below.162  Table  3  identifies  and  connects  the  different  RPD 
componen  ts  of  the  simple  house  fire  scenario. 


161  Klein  et  al.,  “A  Data-Frame,”  132. 

162  Klein,  Sources,  32. 
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Table  3.  Simple  House  Fire 


Cues  recognition 

Pattern  matching 

Mental  models 

One-story  house  in  a  residential 
neighborhood 

Bedrooms/living  areas  on  first 
floor  or  basement 

Extinguishing  fire  allowed  units 
to  search  the  first  floor/basement 
bedrooms 

Fire  is  in  the  back,  in  the  kitchen 
area. 

Food  on  stove,  gas-fueled  fire, 
oven  fire,  electrical  fire,  grease 
fire 

Food  on  stove  became  2nd  alarm 
fire  —  there  are  NO  routine  fires 

The  water  should  have  more  of 
an  impact 

Water  extinguishes  fire  and 
reduces  heat;  black  smoke  turns 
white 

Gas  fueled  fire  required  the  gas  to 
be  turned  off  before  final 
extinguishments. 

Kitchen  fire  that  was  an  extension 
of  a  basement  fire.  Voids  provide 
undetected  extension  of  fire  from 
basement 

Dousing  it  again,  and  get  the 
same  results. 

Additional  water  should 

accomplish  final  extinguishment 

Accelerant  fire  with  inadequate 
water  pressure. 

Perfectly  standard  building  with 
nothing  out  of  the  ordinary 

Sound  buildings  not  subject  to 
collapse 

Basement  fire  burned  undetected, 
and  caused  first  floor  to  collapse; 
water  did  not  reach  seat  of  fire. 

When  dealing  with  an  asymmetrical  threat  environment,  first  responders  will 
often  not  possess  a  mental  model  for  the  situation  at  hand,  nor  the  capacity  for  engaging 
in  pattern  recognition,  and  must  therefore  have  a  contingency  plan.  Mental  simulations, 
rehearsing  mentally  how  a  selected  option  may  play  out  and  comparing  different  courses 
of  action,  is  one  such  step. 

D.  MENTAL  SIMULATION: 

Mental  simulation  is  defined  by  Klein  as  “the  ability  to  imagine  people  and 
objects  consciously  and  to  transform  those  people  or  objects  through  several  transitions 
finally  picturing  them  in  a  different  way  than  at  the  start.”163 

Mental  simulation  comes  into  play  when  the  “simple-match  RPD  model”  fails  to 
intuitively  recognize  a  solution.  Teaching  the  art  of  mental  simulations  involves  mentally 
rehearsing  options  to  determine  workability  for  current  problem  solving,  inputting,  or 
adjusting  as  warranted.  The  development  of  good  mental  simulation  skills  will  improve 
accuracy  and  save  time  in  critical  situations. 


163  Klein,  Sources,  43. 
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By  rehearsing  a  COA,  mental  simulation  helps  the  decision  maker  to  make  sense 
of  events  that  are  void  of  the  critical  cues  and  patterns  that  activate  intuition.164  The 
decision  maker  will  play  out  options  in  their  mind's  eye,  making  use  of  mental  models, 
analyzing  actions  that  must  be  taken,  and  results  of  implementation,  while  identifying 
problems  with  selected  courses  of  action/options  and  their  solution.165  “When  we  use 
mental  simulation  to  derive  a  plausible  explanation,  we  feel  that  we  have  diagnosed  the 
situation  the  way  a  physician  might  diagnosis  a  disease  or  an  auto  mechanic  might 
diagnose  an  engine  problem.”166 


A  fire  commander  responding  to  an  early  morning  alarm  for  an  advanced  fire  in  a 
newly  constructed  commercial  building  must  decide  between  an  interior  or  exterior 
attack.  Among  the  many  cues  the  commander  will  consider  in  his  size  up,  two  or  three  are 
critical  and  are  immediately  apparent;  the  amoun  t  and  volume  of  fire,  time  of  day,  and 
building  stability.  Before  making  a  choice,  the  commander  will  consider  relevant  cues, 
recognize  patterns,  and  compare  and  contrast  the  patterns  to  mental  models  of  previous 
fires.  If  time  permits  or  patterns  are  not  making  sense,  the  commander  may  mentally 
rehearse  a  series  of  actions  predicated  on  cues.  For  example,  the  volume  of  fire  could 
indicate  how  long  the  building  has  been  burning  or  if  the  fire  is  perhaps  arson  related; 
commercial  buildings  are  normally  unoccupied  after  business  hours;  and  newly 
constructed  buildings  are  not  built  to  burn,  while  those  in  the  state  of  disrepair  are  prone 
to  collapse  earlier  than  expected. 


Regarding  the  commercial  building  fire  scenario,  remove  any  information 
indicating  that  the  fire  is  in  the  advanced  stage.  For  some  fire  commanders  (experts)  the 
remaining  two  critical  cues  (time  of  day  and  construction)  may  still  be  enough  to  make  an 


164  Klein,  Sources,  89. 

165  Raanan  Lipshitz,  “Converging  Themes  in  the  Study  of  Decision  Making  in  Realistic  Settings,” 
Decision  Making  In  Action:  Models  And  Methods,  (eds)  Roberta  Calderwood  et  al.,  (Norwood,  New  Jersey: 
Ablex  Publishing  Corporation,  1993):  109. 

166  Ibid.,  89. 
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intuitive  decision  in  favor  of  an  exterior  attack.  The  commander’s  mental  simulations 
concerning  decisions  made  can  be  cognitively  processed  in  numerous  ways.  First,  if  time 
pennits  in  an  intuitive  setting,  the  commander  may  mentally  rehearse  the  decisions  to 
identify  any  unforeseen  problems  before  implementing  action.  He  may  modify  or  reject 
an  option  or  possibly  explore  another  action.167  Second,  when  a  problem  or  a  decision¬ 
making  opportunity  does  not  provide  enough  familiarity  to  be  solved  intuitively,  the 
commander  will  decide  on  a  COA  by  mentally  playing  out  identified  options  to 
determine  their  feasibility  for  success.  If  one  works,  he  will  implement  the  decision;  if 
not,  he  will  choose  another  course  of  action  and  repeat  the  simulation  steps. 

Continuing  with  the  previous  scenario,  imagine  that  the  commander  is 
considering  suggestions  to  implement  an  interior  attack.  Because  there  are  no  mental 
models  available  for  pattern  matching,  he  will  mentally  play  out  how  the  process  will 
unfold.  He  may  envision  the  following: 

•  Optimum  apparatus  placement  for  unit  safety.  Is  it  available? 

•  Proximity  of  command  post  to  building  for  adequate  communication  with 
interior  operations.  Can  communication  be  maintained? 

•  Resources  on  hand  if  plan  needs  to  be  changed.  How  fast  can  they  go  to  an 
exterior  attack  if  needed? 

By  mentally  rehearsing  options,  the  commander  will  validate  or  disprove  their 
COA.  Agreed  upon  decisions  are  not  necessarily  the  best  choice,  although  they  are  a 
workable  solution.  This  is  because,  in  many  highly  time-compressed  decision-making 
situations  such  as  firefighting,  there  is  not  time  to  come  up  with  the  optimal  solution. 
Instead,  decision  makers  use  a  “satisficing”  strategy  to  arrive  at  a  workable  solution  given 
the  time  constraints.168 


167  Gary  A  Klein,  “A  Recognition-Primed  Decision  (RPD)  Model  of  Rapid  Decision  Making,” 
Decision  Making  In  Action:  Models  And  Methods,  (ed)  Roberta  Calderwood  et  al,  (Norwood,  New  Jersey: 
Ablex  Publishing  Corporation,  1993)  140. 

168  Stanley  C.  Collyer  and  Gerald  S.  Malecki,  “Tactical  Decision  Making  Under  Stress:  History  and 
Overview,”  (eds)  Janis  A.  Cannon-Bower  and  Eduardo  Salas,  Making  Decisions  Under  Stress 
(Washington,  DC:  American  Psychological  Association,  1998):  7. 
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A  word  of  caution  concerning  mental  simulation:  The  person  constructing  the 
mental  simulation  needs  to  be  familiar  with  the  particular  task,  thinking  at  the  right  level 
of  abstraction.  You  cannot  mentally  simulate  an  option  foreign  to  your  domain  of 
expertise.  For  example,  a  fire  chief  has  no  background  knowledge  regarding  the  mental 
cognitions  of  a  neonatal  nurse  rehearsing  emergency  interventions  of  a  newborn  in 
distress. 

Mental  simulations  do  not  need  to  be  elaborate.  Due  to  the  limits  of  our  working 
memory,  six  steps  should  be  the  limit.  If  there  are  too  many  transitions,  the  decision 
maker  will  lose  track.169  When  span  of  control  becomes  an  issue,  the  decision  maker  can 
“chunk”  several  transitions  into  one  unit.  If  the  simulation  is  too  detailed,  it  will  take  up 
critical  working  memory  space.  The  decision  maker  can  assume  certain  steps  being  taken 
without  actually  thinking  it  out.  This  presumption  is  a  good  example  of  SA  automatic 
action  selection.  A  simulation  comprised  of  intangibles  cannot  be  measured,  therefore  a 
certain  amount  of  domain  knowledge  is  necessary  to  separate  fact  from  fiction.170 

The  decision  maker  must  keep  in  mind  that  simulations  can  be  wrong;  “they  are  a 
means  of  creating  explanations,  not  for  generating  proofs.”171  While  mental  simulation  is 
effective  when  the  decision  maker  is  not  quite  sure  what  is  happening,  trying  to  puzzle 
out  the  problem;  time-pressure,  and  too  many  moving  parts  can  complicate  their  puzzle 
and  affect  the  quality  of  mental  simulation.172 

In  summary,  filtered  information  from  the  environment  activates  the  mental 
model,  creating  numerous  situational  models  designed  to  assist  decision  making. 
Selective  cues  help  the  decision  maker  to  recognize  patterns,  activating  action  scripts  in 
the  fonn  of  mental  simulations.  Mental  simulations  rely  on  stored  mental  models  to 


169  Klein,  Sources,  52. 

170  Ibid.,  53. 

171  Ibid.,  68. 

172  Ibid.,  69. 
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mentally  rehearse  different  scenarios  to  detennine  if  the  decision  maker’s  choice  will  be 
effective.  “In  order  to  build  an  effective  mental  simulation,  we  need  to  have  good  mental 
models  of  how  things  work.”173 


173  Klein,  Power.  27. 
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IV.  DECISION-MAKING  STRATAGEM 


Chapter  IV  outlines  a  stratagem  to  transform  first  responders  into  better  decision 
makers  when  confronting  uncertainty.  Earlier  chapters  described  SA  and  the  RPD  model 
strategies  of  cue  recognition,  pattern  matching,  mental  models,  and  mental  simulations. 
Chapter  IV,  however,  identifies  the  characteristics  that  constitute  an  effective  decision¬ 
making  strategy  in  the  following  ways:  first,  through  identification  and  evaluation  of  the 
mechanisms  of  NDM  and  expertise;  second,  by  categorizing  trainable  decision-making 
skills  and  the  methods  for  training  them;  and  finally,  by  suggesting  a  scenario-based 
experiment  to  detennine  if  first-responder  decision-making  perfonnance  in  an 
asymmetrical  threat  environment  can  be  enhanced. 

NDM  strategies  enhance  an  individual’s  intuitive  decision-making  skills,  while 
SA  fast  tracks  proficiency  and  expertise  development  for  a  novice  decision  maker’s 
training.  While  training  has  been  and  will  continue  to  be  considered  the  educational 
methodology  of  choice,  intended  to  enhance  decision  making,  it  may  not  be  the 
appropriate  word  to  describe  this  process.  Two  issues  are  paramount  regarding  a  person 
becoming  more  skilled  in  their  domain:  First,  educators  may  want  to  consider  the  process 
more  of  an  education  than  training  regimen;  “training  provides  specific  skills,  whereas 
education  provides  the  basis  for  further  skill  acquisition.”174  Second,  Klein  tells  us  that 
training  a  novice  to  think  like  an  expert  is  too  time-consuming  and  expensive.  However, 
maybe  we  can  get  a  novice  to  learn  like  an  expert.175  As  discussed  earlier,  a  novice  can 
learn  like  an  expert  by  engaging  in  deliberate  practice,  compiling  an  extensive  experience 
bank,  and  obtaining  accurate  feedback  on  decisions  made.176 

Before  beginning  an  educational  undertaking  designed  to  prepare  novice  decision 
makers  to  leam  like  experts  in  asymmetrical  threat  environments,  the  “end  to  the  means” 
regarding  the  characteristics  that  underlie  effective  decision  making  must  be  detennined. 

174  Stoney  A.  Trent,  “Team  Cognition  in  intelligence  analysis  training,”  Dissertation  (Ohio  State 
University,  2007):  21. 

175  Klein,  Sources,  104. 

176  Ibid. 
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In  other  words,  what  should  they  be  able  to  do?  While  there  are  numerous  skills  and 
training  methods  for  enhancing  decision-making  effectiveness,  the  following  table 
outlines  the  characteristics  (knowledge  and  skill)  and  mechanisms  of  NDM  that  enhance 
decision-making  performance  in  an  asymmetrical  environment  —  those  decision-making 
skills  and  methods  specific  to  this  study.  By  doing  so,  educators  will  be  better  prepared  to 
make  the  transition  from  “what  to  train”  (skills)  to  “how  to  train  it”  (methods). 


Table  4.  NDM  Characteristic  and  Mechanism/Expertise 


Characteristics 

Mechanism/Expertise 

•  Flexibility/adaptability 

•  Quick/resilient 

•  Risk-taker/accurate 

•  Situational-Awareness  skills 

•  Organized  knowledge  structure 

•  Mental  simulation,  models,  cue/pattern 
recognition 

•  Reasoning  skills 

A  WHAT  TO  TRAIN  (SKILLS) 

Situational  Awareness  Skills 

•  Ability  to  conduct  a  rapid  accurate  assessment;  synthesis  of  perception, 
comprehension,  and  projection  SA  Levels. 

Cue  recognition/pattem-matching  skills. 

•  Trusting  intuitive  decision  making 
Mental  Simulation  Skills 

•  The  art  of  rehearsing  a  solution 
Mental  Models 

•  Selecting  the  right  model  for  the  right  situation/constructing  templates 
Domain-Specific  Problem-Solving  Skills 

•  Problem  solving  in  domain  of  expertise,  not  generic 
Reasoning 

•  Critical  thinking  and  creative  problem  solving/improvising 
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B.  HOW  TO  TRAIN  (METHODS) 


Design  scenario  based  training,  i.e.,  computer  based  simulations,  paper/pencil 
decision  skill  training,  tabletop  exercises... 

•  Domain-specific  scenarios  allow  the  trainee  to  buy  into  the  hypothesis  of  the 
situation.  If  individuals  can  see  the  relevance  of  the  training  to  their  job,  learning 
will  take  place.  “The  most  important  criteria  concerning  simulations  are  whether 
the  decision  maker  takes  the  scenarios  seriously.”177 

•  Scenario-based  training  can  be  the  conduit  for  exposing  individuals  to  the  type  of 
situations  they  will  confront  in  real-world  settings.  Unlike  real  world  events, 
however,  and  full-scale  exercises,  scenarios  can  be  stopped,  paused,  and  rewound. 
Mastered  skills  can  be  fast-forwarded,  while  remedial  training  can  be  conducted 
on  those  skills  not  yet  perfected.  For  example,  scenario  simulations  are  safer  than 
full-scale  exercises,  not  exposing  individuals  to  unnecessary  risks. 

•  Scenario-based  simulations  provide  more  practice  trials;  the  very  problem-solving 
skills  that  the  trainer  is  trying  to  enhance  —  namely  situational  awareness,  cue 
recognition,  pattern  match,  mental  models,  and  mental  simulation  —  are 
strategically  included  in  the  scenario.  These  skills  can  be  enhanced  through 
repeated  exposure  (deliberate  practice)  and  designing  scenarios  around  training 
objectives.  For  example,  displaying  cues  with  associated  patterns  will  help  to 
develop  templates  (mental  models)  for  future  use.178  Because  opportunities  to 
develop  decision-making  expertise  via  real-world  events  are  limited,  scenario- 
based  training  provides  a  chance  to  acquire  skills  that  real-world  events  may  not 
have  the  opportunity  provide. 

When  considering  “what  to  train,”  educators  need  to  make  certain  the 
instructional  design  is  context-specific  domain  knowledge.  To  make  training  truly 
effective,  individuals  need  to  work  within  their  area  of  expertise.  For  example,  if  the  goal 

177  Gary  A.  Klein  and  David  D.  Woods,  “Conclusion:  Decision  Making  In  Action”  (eds)  Roberta 
Calderwood  et  al.,  Decision  Making  In  Action:  Models  And  Methods  (Norwood,  New  Jersey:  Ablex 
Publishing  Corporation,  1993):  410. 

178  Canon-Bowers,  “Training  Decision  Makers  for  Complex  Environments:  Implications  of  the 
Naturalistic  Decision  Making  Perspective.”  105. 
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is  to  enhance  a  fire  officer’s  decision-making  capabilities  while  operating  in  areas  of 
uncertainty,  scenario-based  instructions  should  begin  by  introducing  the  officer  to  an 
environment  comprised  of  familiarities,  one  in  which  they  feel  somewhat  comfortable 
regarding  incident  problem  solving.  Then,  once  familiarity  is  established,  strategically 
tweak  the  scenario  with  situational  injects  that  develop  uncertainty.  This  is  an  ideal 
opportunity  to  evaluate  JIT  mental  models  —  the  use  of  fragmentary  knowledge  to 
activate  good  mental  models.179  Do  novice  officers  possess  enough  domain  knowledge  to 
enable  them  to  piece  together  a  good  synopsis  regarding  the  uncertainty?  Chapter  V 
explains  the  scenario-based  training  methodology,  while  describing  the  thesis  experiment 
in  detail. 


179  Klein  et  at,  “A  Data-Frame,”  130. 
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V.  SCENARIO-BASED  LEARNING  AND 
EXPERIMENTAL  DESIGN 


We  have  discussed  scenario-based  examples  as  a  means  to  provide  exposure  to 
asymmetric  scenarios  that  first  responders  do  not  experience  in  their  normal  jobs. 
Scenario-based  training  provides  an  effective  way  to  expose  practitioners  to  events  and 
let  them  respond  when  there  are  not  opportunities  in  every  day  practice.  The  author’s 
goal  here  is  to  reinforce  the  decision  maker’s  thought  process  by  creating  a  skill  and 
application  connection. 

Scenario-Based  Training  (SBT)  entails  the  development  of  cue-driven  exercises. 
The  objective  is  to  expose  trainees  to  response  environments  similar  to  those  in  their 
professional  lives,  while  providing  feedback  on  responses.  According  to  Cannon-Bowers 
et  ah,  SBT  helps  accelerate  the  learning  process  for  the  decision  maker  from  novice  to 
expert  status  by  providing  deliberate  practice  with  a  series  of  task-related  episodes. 
Deliberate  practice  builds  familiarity.  SBT  also  provides  trainees  with  an  opportunity  to 
work  with  equipment  that  is  often  not  readily  available  in  a  controlled  laboratory 
setting.180 

According  to  Ayers  et  ah,  SBT  follows  a  perfonnance-based  imperative.  It  is 
different  from  traditional  training  because  scenarios  incorporate  a  specific  decision¬ 
making  component.  181  SBT  must  focus  on  the  outcome  and  how  it  can  be  improved, 
rather  than  on  the  transfer  of  skills  or  the  acquisition  of  knowledge.  While  skills  are 
imperative  to  SBT,  they  are  attained  via  traditional  training  and  through  experience. 
Scenario-based  exercises  immerse  the  decision  maker  in  situations  requiring 
performance. 

180  John  J.  Bums,  Jams  A.  Cannon-Bowers,  Eduardo  Salas,  John  S.  Pruitt.  ‘‘Advanced  Technology  in 
Scenario-Based  Training”  (eds)  Janis  A.  Cannon,  Eduardo  Salas,  Making  Decisions  Under  Stress. 
(Washington,  D.C.:American  Psychological  Association,  2006):  365. 

181  Frank  Ayers,  MS,  Thomas  Connolly,  Ed.D,  Charles  Robertson,  Ph.D.  Michele  M.  Summers,  MS 
“Managing  Risk  through  Scenario  Based  Training,  Single  Pilot  Resource  Management,  and  Learner 
Centered  Grading”  (September  2007):  8. 

http://www.faa.gov/education  research/training/fits/guidance/media/RM  thorugh  SBT. pdf.  [Accessed 
December  23,  2007]. 
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SBT  is  the  next  best  thing  to  reality;  “Reality  is  the  ultimate  learning  situation  and 
scenario-based  training  attempts  to  get  as  close  as  possible  to  this  ideal.”182  SBT 
incorporates  effective  real-life  scenarios  coupled  with  expert  feedback,  providing  an 
opportunity  for  a  novice  to  learn  like  an  expert.  “In  other  words,  we  leam  better  the  more 
realistic  the  situation  is,  and  the  more  we  are  counted  on  to  perform.”183  Washington 
State  homeland  security  stakeholders,  when  asked  for  their  suggestions  on  improving 
training  materials,  cited  real  life  and  recent  examples  wherever  possible.184  According  to 
the  Homeland  Security  Institute  Report  to  the  WA  State  Committee  on  Homeland 
Security,  the  most  difficult  challenge  regarding  training  is  relating  knowledge  from 
training  to  “real  life.”  Real-life  scenarios  help  participants  apply  what  they  know;  two  by¬ 
products  of  this  application  are  the  use  of  judgment  and  skill. 

A.  EXPERIMENT  HYPOTHESIS 

Through  the  utilization  of  scenario-based  instruction,  this  experiment  attempted  to 
determine  whether  the  decision-making  skills  of  newly  promoted  fire  officers  could  be 
enhanced  through  specific  training  designed  to  develop  those  skills.  The  training 
consisted  of  practice  with  asymmetric  terrorist  and  natural  disaster  scenarios,  expert 
feedback,  and  a  brief  lecture  on  SA  /  RPD  model  strategies. 

By  manipulation  of  the  independent  variables  (SA  /  RPD  brief  and  expert 
feedback),  the  author  attempted  to  achieve  the  desired  outcome  of  improving  novice 
decision-making  skills  (dependent  variable).  In  other  words,  feedback  and  SA  /  RPD 
reinforcement  (independent  variables)  should  have  improved  novice  decision  makers’ 
problem-solving  and  decision-making  skills  in  the  final  scenario.  The  end  goal  was  to 
determine  if  novice  decision-making  skills  could  be  enhanced,  thereby  training  them  to 
learn  like  an  expert. 

182  Ayers  et  al.,  “Managing  Risk  through  Scenario  Based  Training.” 

183  Ibid. 

184  Emergency  Responder  Training  Assessment  and  Recommendations,”  Homeland  Security  Institute 
Report  to  the  WA  State  Committee  on  Homeland  Security  (December  2005):  14. 
http://www.metrokc.gov/prepare/docs/2006-02- 

23%20001%20Emergency%20Responder%20Training%20Assessment%20and%20Recommendations.pdf. 
[Accessed  December  23,  2007]. 
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B. 


GROUP  DESIGN 


Three  groups  were  assembled:  control,  expert,  and  experimental.  The  experts 
were  fourteen  New  York  City  Fire  Department  officers  with  extensive  job  knowledge  and 
decision-making  experience,  identified  by  their  current  rank,  years  of  service,  and  self- 
reported  domain  expertise.  The  experts  had  a  minimum  of  fifteen  years  of  fire  department 
experience  and  at  least  one  year  in  their  current  rank.  The  experimental  and  control 
groups  consisted  of  twenty-nine  New  York  State  Fire  Officers  promoted  within  the  past 
twelve  months.  While  the  use  of  a  random  lot  selection  would  ensure  that  both  groups 
were  of  equal  distribution,  group  designation  (experimental/control)  was  detennined  by 
the  availability  of  participants.  The  control  group  consisted  of  fifteen  officers  from 
various  fire  departments  located  throughout  New  York  State,  excluding  the  FDNY.  The 
experimental  group  was  fourteen  recently  promoted  FDNY  lieutenants.  While  the  period 
of  time  spent  in  previous  rank  (firefighter)  varied,  officer  field-experience  for  the 
experimental  and  control  groups  was  limited  or  in  some  cases  non-existent. 

C.  PRE-EXPERIMENT  QUESTIONNAIRE 

Prior  to  exercise  commencement,  each  group  completed  a  brief  questionnaire 
giving  greater  insight  into  their  fire  service  experience  and  additional  domain-specific  job 
knowledge.  The  expert’s  mean  score  of  years  of  service  with  the  fire  department  (25)  was 
two  times  more  than  the  experimental  (9)  and  control  (12)  groups,  while  experts  years  in 
current  rank  (5)  was  five  times  higher.  In  addition  to  confirming  their  status  as  an  expert 
or  novice,  information  was  collected  on  their  familiarity  with  SOPs.  The  pre-experiment 
questionnaire  asked  groups  to  rate  their  knowledge  regarding  fire  department  standard 
operating  procedures  (SOP)  at  certain  types  of  responses,  while  avoiding  the  risk  of 
predisposing  subjects  to  certain  assumptions,  namely  a  terrorist  attack.  For  example,  a 
terrorist-related  scenario  depicts  an  explosion  at  a  subway  station,  the  questionnaire  asked 
about  subway/transit  SOPs  and  not  about  an  explosives  or  terrorism-related  response.  The 
experts  rated  themselves  higher  than  the  experimental  and  control  groups  in  the  five- 
knowledge/experience  categories.  The  below  average  scores  of  the  experimental  and 
control  group  solidifies  their  status  as  novice  decision  makers  (see  Appendix  A). 
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D. 


THESIS  EXPERIMENT 


The  experiment  began  by  providing  the  expert  group  with  the  one -page  brief 
describing  SA  and  RPD  strategies.  This  description  established  a  connection  between 
decision-making  theory  and  expert  practical  application  by  providing  real  world 
analogies  (see  Appendix  B).  Next,  a  written  practice  scenario  depicting  an  emergency 
response  to  a  novel  situation,  i.e.,  a  terrorist  scenario,  was  distributed  to  each  member  of 
the  expert  group.  The  researcher  read  the  scenario  passage  aloud  to  a  selected  decision 
point  (participants  could  read  along  silently).  At  each  decision  point,  the  experts  were 
asked  one  to  three  questions  in  several  different  formats.  For  example,  one  format  asked 
the  expert  group  to  prioritize  from  a  list  of  possible  options,  while  another  fonnat  asked 
the  group  to  provide  a  yes  or  no  response.  The  lists  of  possible  actions/options  to  decision 
point  questions  were  developed  from  five  pre-experiment  scenario  pilot  sessions  with 
other  peer  experts.185 

A  scenario  response  key  was  developed  by  considering  the  experts’  most  common 
responses  as  correct  answers  to  decision  point  questions.  For  example,  if  twelve  out  of 
fifteen  experts  provided  the  same  response  to  a  question,  that  response  was  considered 
correct.  For  responses  requiring  experts  to  prioritize  from  a  list  of  six  to  seven  actions, 
the  top  three  prioritized  responses  became  the  answer  key.  The  experts  were  also  asked  to 
explain  their  decision-making  process  and  problem-solving  strategies  in  the  context  of 
SA  and  RPD,  the  why  behind  their  decision  making  and  prioritization.  These  responses 
provided  the  foundation  for  developing  feedback  for  the  experimental  group.  For 
example,  an  underlining  theme  throughout  the  scenario  readings  was  the  protection  of 
first  responding,  making  sure  they  do  not  become  part  of  the  problem  by  becoming 
victims  themselves.  One  expert  provided  the  following  analogy  to  reinforce  the  need  to 
protect  first  responders  first: 

185  At  the  pre-experiment  sessions,  the  scenarios  were  read  to  other  experts  in  the  same  fashion.  Their 
responses  were  captured,  summarized,  and  studied  for  commonalities  between  group  members.  Common 
responses  were  developed  into  actions  or  decision-making  option  lists  unique  to  each  scenario  decision 
point  questions.  These  pilot-sessions  turned  out  to  be  invaluable  during  the  actual  experiment  because  the 
groups  were  given  2.5  minutes  to  make  decisions  and  provide  explanations  when  asked.  By  asking  the 
individuals  to  prioritize  from  a  list  of  responses  designed  solely  to  that  specific  passage,  all  three  groups 
were  considering  the  same  information  and  suitable  responses. 
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Part  of  an  airline  pre-flight  safety  plan  prior  to  take-off  includes  instructions  regarding 
the  donning  of  individual  breathing  masks  —  in  case  cabin  pressure  is  lost  while  the 
plane  is  in  the  air.  Individuals  with  a  child  are  told  to  don  their  mask  first  and  then  the 
child’s.  Why?  Because,  if  conditions  require  supplementary  oxygen,  you  may  be 
overcome  while  trying  to  outfit  your  child.  In  which  case,  what  good  would  you  be  to 
your  child?  The  same  analogue  goes  for  firefighting.  Managers  need  to  make  decisions 
ensuring  their  personnel  are  properly  equipped  before  sending  them  into  danger  ...  we 
must  protect  ourselves  first  ...  and  this  involves  decision  making  ...  possibly  unlike  those 
of  the  past  because  of prior  knowledge/experience. 


This  procedure  continued  until  all  the  scenario  passages  were  read  and  scenario 
decision  points  completed.  A  total  of  four  asymmetric  type  scenarios  (tanker  on  bridge, 
subway  explosion,  radiological  attack,  and  earthquake)  were  read  to  the  expert  group, 
with  three  practice  and  one  actual  experimental  test  scenario  (see  Appendix  C,  D,  E,  F) 
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Figure  5.  Expert  Group  Experiment  Matrix 

The  same  scenarios  were  distributed  to  the  control  and  experimental  groups.  Each 
scenario  passage  was  read  aloud  to  the  decision  point  (participants  could  read  along 
silently).  At  each  decision  point,  the  researcher  asked  one  to  three  questions  and  then 
gave  participants  2.5  minutes  to  provide  a  response.  This  procedure  continued  until  all 
the  scenario  passages  were  read  and  scenario  decision  points  completed. 

Upon  completion  of  each  scenario  decision  point,  the  experimental  group  was 
exposed  to  the  independent  variables  (SA  /  RPD  brief  and  experts’  feedback).  The  control 
group  was  not  exposed  to  the  independent  variables.  The  goal  of  expert  feedback  was  to 
improve  the  decision  making  of  the  experimental  group  at  subsequent  scenario  decision 
points  based  on  providing  examples  of  how  experts  responded  to  other  scenarios.  The 
hypothesis  was  that  providing  examples  of  the  way  experts  responded  to  similar  types  of 
scenarios  would  persuade  the  experimental  group  to  learn  and  make  decisions  like  the 
experts,  considering  and  weighing  options/actions  they  may  not  have  in  the  past. 
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Figure  6.  Experiment  /  Control  Group  Experiment  Matrix 
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At  selected  decision  points  (one  per  scenario),  the  experimental  group  completed 
a  decision  point  questionnaire  regarding  decisions  made  (see  Appendix  G).  These 
decision  points  were  selected  because  they  provided  the  opportunity  for  researchers  to 
capture  the  effects  of  expert  feedback  and  the  SA  /  RPD  model  brief  on  groups’  decision 
making  in  real  time,  during  the  scenario.  On  a  scale  of  1-5,  with  1  being  low  and  5  high, 
the  experimental  group  rated  the  following:  The  decision  difficulty  and  level  of 
confidence  regarding  the  selected  decision  point.  The  role  that  experience  (mental 
models),  rehearsal  (simulation),  and  pattern  matching/cue  recognition  played  in  decision 
making  and  problem  solving.  Additionally,  the  three  levels  of  situation  awareness, 
perception  (size-up),  comprehension  (making  sense)  and  projection  (predicting  what  will 
happen  next)  were  also  rated.  The  control  group  did  not  complete  this  questionnaire  since 
they  did  not  receive  the  independent  variables  treatment. 

E.  EXPERIMENTAL  /  CONTROL  GROUP  SCORING 

First  responders  often  multi-task  at  emergency  incidents,  making  several 
decisions  simultaneously  or  in  consecutive  fashion,  therefore,  when  prioritizing  from  a 
list  of  potential  actions,  the  individual’s  top  three  responses  that  agreed  with  the  expert 
response  key  regardless  of  order/ranking  were  scored  as  correct.  For  example,  an 
individual  might  select  the  same  three  responses  as  the  expert  group  but  rank  them  in  a 
different  top  order.  The  response  will  be  considered  correct  and  they  will  receive  full 
credit.  However,  if  the  individual’s  response  matches  only  two  of  the  three  answer  key 
response,  they  received  partial  credit  for  that  decision  point.  Table  4  was  taken  from  the 
earthquake  scenario  decision  point  #  4.6.  At  this  decision  point,  the  experimental  and 
control  group  members  were  read  the  following  passage  and  asked  to  prioritize  a  list  of 
six  possible  actions.  When  compared  against  the  expert  response  key,  both  members 
received  partial  credit  for  their  responses. 
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Table  5.  Sample  Earthquake  Scenario  Passage 


Scenario:  Earthquake  4 

Upon  arrival  you  observe  someone  lying  face  down  in  front  of  the  station  with  two  men  running  from 
the  location,  opposite  your  approach. 

•  The  streets  in  and  around  the  station  are  crowded  with  people  either  requesting  help  or  seeking 
information  about  the  initial  earthquake. 

•  Incident  size-up  reveals  gasoline  leaking  from  several  fuel  pumps  with  fuel  pooling  in  and 
around  the  station  and  street. 

•  Gasoline  has  also  been  discovered  coming  up  through  the  ground  in  the  rear  of  the  station.  This 
fuel  is  flowing  down  the  street  adjacent  to  the  curb  line  and  smoke  is  venting  under  pressure 
from  the  front  cornice  of  the  station. 


Scenario  Decision  Point  #  4.6: 

•  Prioritize  your  decision-making.  What  problems  do  you  address  first,  second,  third...?  Explain 
your  first  three  decisions. 

_ Fuel  coming  from  rear  of  station  and  flowing  down  the  street 

_ Person  lying  face  down 

_ Smoke  from  cornice 

_ Fuel  pooling  in  front  of  station 

_ People  requesting  help/information 

_ Lack  of  resources  (foam/EMS) 


Expert  group's  prioritization  response  key 


Ranking 

Prioritize  Decision  Making 

1 

Person  lying  face  down 

2 

Fuel  coming  from  rear  of  station  and  flowing  down  the  street 

3 

Smoke  from  cornice 

Experimental  group  member 


Ranking 

Prioritize  Decision  Making 

1 

Person  lying  face  down 

2 

Lack  of  resources  (foam/EMS) 

3 

Smoke  from  cornice 

Control  group  member 


Ranking 

Prioritize  Decision  Making 

1 

Lack  of  resources  (foam/EMS) 

2 

Smoke  from  cornice 

3 

Person  lying  face  down 
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Expert  Feedback 

Person  lying  face  down  is  a  known  life  hazard  (visual  confirmation)  and 
addressing  life  is  a  #  1  priority  before  property.  Unit  can  make  quick  assessment  and 
determine  viability  of  victim  removal.  The  person  lying  down  may  possibly  be  victim  of 
robbery  and  requiring  immediate  help. 

Fuel  coming  from  rear  of  station  and  flowing  down  the  street  needs  to  be 
controlled.  If  flow  cannot  be  stopped,  units  must  make  an  attempt  to  contain  or  divert 
flow  from  reaching  an  ignition  source.  Leaking  fuel  is  largest  threat  to  life/property,  if 
ignited  it  will  intensify  an  already  dynamic  and  developing  situation. 

Extinguishing  the  fire  eliminates  a  major  problem.  For  example,  smoke  from  the 
fire  has  potential  to  expose  hospital  and  apartment  building.  Investigating  the  source  of 
smoke  will  reveal  fire  intensity,  determining  if  a  quick  stop  of  fire  can  be  accomplished 
or  whether  operating  personnel  should  be  moved  to  a  safer  area 


The  earthquake  scenario  had  eight-scenario  decision  points  that  added  up  to  100 
percent.  Therefore,  each  decision  point  was  worth  between  ten  and  fifteen  points.  The 
researcher  detennined  decision  point  weight  by  considering  the  difficulty  of  the  response, 
impact  on  the  scenario  outcome,  and  number  of  decision  point  questions.  Scenario 
decision  point  #  4.6  was  worth  fifteen  percent  of  total  earthquake  scenario  score; 
therefore  both  individuals  received  credit  for  ten  percent  because  they  were  correct  in  two 
of  the  three  responses. 

F.  SUMMARY 

Upon  completion  of  the  four  scenarios,  the  experimental  and  control  groups 
provided  feedback  by  completing  the  post-experiment  questionnaire  (see  Appendix  H). 
The  goal  of  this  experiment  was  to  detennine  if  the  decision-making  and  problem¬ 
solving  skills  of  newly  promoted  fire  officers,  when  operating  in  an  asymmetric  threat 
environment  and  without  experience,  could  be  enhanced  through  specific  training 

designed  to  develop  those  skills.  If  it  can  be  enhanced  through  training,  then  the  analysis 
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of  scenario  decision  making  amongst  the  three  groups,  comparing  the  experimental  and 
control  group’s  scenario  responses  with  those  of  the  expert  groups  should  indicate  a 
significant  difference.  While  this  experiment  provided  greater  insight  into  the  hypothesis 
on  improving  first  responder  decision  making  in  an  asymmetric  enviromnent,  Chapter  VI 
analyzes  group  response  data  while  making  recommendations  based  on  experiment 
findings. 
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VI.  ANALYSIS  AND  RECOMMENDATIONS 


To  determine  the  impact  of  the  independent  variables  (expert  feedback  and  SA  / 
RPD  briefing),  a  one-way  ANOVA  was  used  to  compare  the  earthquake  scenario  scores 
of  the  experimental  and  control  groups  (see  Appendix  I).  Contrary  to  its  namesake,  an 
ANOVA  is  concerned  with  differences  between  mean  scores  and  not  between  variances. 
A  one-way  ANOVA  looks  to  see  if  there  is  a  variation  between  group  means.186  When 
the  difference  is  larger  than  would  normally  be  expected,  you  have  a  significant 
difference.187  The  ANOVA  “F”  value  of  14.087  yielded  a  “p”  value  of  .001(the 
significance  is  greater  than  .05).  Therefore,  we  can  say  with  confidence  that  there  was  a 
significant  difference  between  the  experimental  and  control  groups’  mean  scores  for  the 
final  asymmetrical  scenario  (earthquake),  or  the  two  groups  are  statistically  different 
from  each  other  (see  Appendix  J). 

Furthermore,  since  the  distinction  between  the  two  groups  throughout  the  course 
of  the  four  scenario  readings  was  the  experimental  group’s  receipt  of  independent 
variables  SA  /  RPD  briefing  and  expert  feedback,  one  concludes  that  the  inclusion  of  the 
independent  variables  helped  to  achieve  the  dependent  variable,  namely,  improving  first- 
responder  decision  making  in  an  asymmetrical  threat  environment. 

A.  RECOMMENDATION 

Experiment  findings  indicated  that  novice  decision  making  was  influenced  by 
external  factors.  When  comparing  the  control  and  experimental  groups’  scores  on  the 
earthquake  scenario,  novices  in  the  experimental  groups  were  beginning  to  respond  like 
experts.  Expert  feedback  that  was  provided  during  practice  scenarios  was  paying 
dividends,  an  exchange  of  knowledge  had  taken  place;  novice  and  expert  were 
considering  the  same  cues  in  their  decision  making,  coming  to  similar  conclusions. 

186Will  G  Hopkins,  “T-Test  and  One-  Way  ANOVA,”  A  New  View  of  Statistics,  2000. 
http://www.sportsci.org/resource/stats/ttest.html. 

187  Susan  Archambault,  “One-way  ANOVA,”  Psych  205  Review  Site,  Psychology  Department, 
Wellesley  College.  2002.  http://www.welleslev.edu/Psvchologv/Psych205/anova.html. 
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Several  questions  surface  regarding  the  thesis  hypothesis  and  experiment  finding. 
First  and  foremost,  “How  can  first  responder  agencies  use  experiment  findings  to 
improve  personnel  decision  making?”  In  other  words,  how  can  they  get  their  novices  to 
learn  like  experts?”  There  is  no  need  to  re-invent  the  training  wheel  here.  The  training 
level  of  application,  however,  may  need  to  be  elevated.  For  example,  whereas  classroom 
instruction  often  involves  the  more  traditional  instructor-led  discussions,  experience  and 
naturalistic  decision-making  expertise  may  be  best  conveyed  to  the  novice  via  scenario- 
based  exercises  that  provide  expert  feedback  regarding  decisions  made. 

While  designers  of  scenario-based  exercises  should  strive  to  focus  on  domain- 
specific  situations  that  pair  expert  and  novice  decision  makers  together  in  unfamiliar 
circumstances  and  have  them  problem  solve;  this  is  not  always  a  requisite  of  an  effective 
scenario.  For  example,  the  premise  that  drove  the  earthquake  scenario  was 
inconsequential.  It  could  have  easily  been  a  hurricane-  or  flood-driven  scenario  and  still 
have  been  effective.  Although  decision-making  situations  should  be  domain-specific 
whenever  possible,  scenarios  merely  provide  a  reason  for  the  decision  makers  to  be  there, 
and  they  should  not  always  be  designed  with  the  sole  purpose  of  testing  subject  matter 
experts.  In  the  case  of  the  earthquake  scenario,  observing  the  exchange  of  information 
between  experts  and  novices  —  with  the  intent  of  developing  better  decision  makers  by 
having  experts  explain  the  why  behind  decisions  made  —  was  a  primary  objective. 

For  example,  scenario-based  exercises  need  to  emphasize  not  only  what  experts 
consider  relevant,  but  also  why.  In  other  words,  show  the  math.  If  the  experts  provide  the 
answer  without  explanation,  will  the  learning  be  short-term?  When  developing  scenario- 
based  exercises,  designers  must  be  certain  that  experts  expand  upon  their  response  by 
describing  in  detail  the  cues,  patterns,  and  experiences  (mental  models)  they  considered 
when  making  connections.  Did  they  mentally  puzzle  out  a  COA  before  coming  to  a 
decision?  What  was  considered  in  their  SA  analysis?  Otherwise,  minus  expert  guidance, 
the  novice  gains  no  advantage  when  confronting  an  unfamiliar  situation.  The  scenario 
should  drive  the  exercise,  not  the  decision  making.  Because,  in  reality,  there  may  be  no 
real  experts. 
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So,  if  there  are  no  or  few  bona  fide  experts  when  situations  are  novel,  how  does 
the  expert  get  by?  The  complexities  that  constitute  an  asymmetrical  threat  environment 
often  turn  experts  into  novices  for  that  particular  situation,  and  yet  experts  can  make 
sound  decisions.  This  statement  was  corroborated  when  experts,  who  rated  their  natural 
disaster  knowledge/experience  levels  barely  above  average,  puzzled  out  the  unfamiliar. 
That  capacity  —  solving  a  novel  situation  when  information  is  limited  —  is  what  experts 
need  to  convey  to  novices.  There  are  several  schemes  for  designing  this  higher  first- 
responder  decision-making  training  curriculums. 

As  a  fire  officer  advances  through  the  various  promotional  ranks,  he  is  often 
required  to  attend  officer  training  before  moving  to  a  subsequent  level.  One  objective  of 
this  orientation  is  to  familiarize  officers  with  the  decision-making  and  problem-solving 
responsibilities  of  the  new  rank.  For  larger  departments,  promotional  courses  may  be 
conducted  simultaneously,  with  newly  promoted  lieutenants,  captains,  and  chiefs  each 
attending  individual  orientation  training  courses.  Experienced  fire  officers  are  leaving 
familiar  roles  regarding  decision-making  responsibilities,  and  preparing  to  assume  new 
roles  with  novel  decision-making  opportunities.  How  can  departments  harness  this  expert 
knowledge  before  it  leaves  the  rank? 

Scenario-based  simulation  exercises,  bringing  officers  of  different  ranks  together, 
may  be  one  answer.  This  would  involve  having  senior  incumbent  members  preparing  to 
leave  their  current  rank  work  through  scenario-based  exercises  with  less  experienced 
officers  filling  the  created  vacancy.  This  process  is  cyclical  because,  as  the  “tricks  of  the 
trade”  are  conveyed  between  incoming  and  outgoing,  the  officer  is  both  beneficiary  and 
benefactor  of  vital  decision-making  information.  Imagine  a  group  of  captains  preparing 
for  their  new  rank  by  working  alongside  a  group  of  senior  chiefs  managing  a  large-scale 
hazardous  materials  incident  via  a  three-hour  scenario-based  simulation,  and  then  have 
those  same  captains  school  a  group  of  lieutenants  regarding  the  company  commander 
responsibilities  inherited  when  assuming  the  captains’  rank. 

Decision  making  in  today’s  asymmetrical  environment  is  as  much  a  concern  for 
the  newly  promoted  first-line  supervisor  responding  as  a  boss  to  an  incident  with  terrorist 

implications  as  it  is  for  a  three-star  chief  with  over  twenty-five  years  of  service  assuming 
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command  of  an  escalating  incident  with  no  known  cause  or  conclusion.  Maybe  there  are 
no  “real  technical  experts”  for  asymmetrical  incidents  just  yet,  but  rather  individuals  with 
good  SA  /  decision-making  skills.  Individuals  with  the  ability  to  make  sense  of  their 
surroundings,  using  what  little  cues  and  patterns  the  environment  offers.  Individuals  that 
can  route  decisions  through  mental  rehearsals  of  plausible  COA,  project  future  events, 
and  provide  expert  feedback  on  decisions  made.  Sounds  insurmountable?  Well  it  is  not. 


While  participating  as  an  expert  in  the  subway  explosion  scenario,  a  well- 
respected  FDNY  battalion  chief  with  more  than  thirty  years  of  fire  department  sendee 
needed  to  make  a  decision  regarding  the  stopping  of  trains  while  operating  at  a 
suspected  terrorist  attack  upon  a  subway  platform.  Having  no  mental  models  with 
subway  incidents,  he  created  an  analogy  with  high-rise  building  fires,  with  which  he  had 
a  fair  amount  of  firefighting  experience.  Sounds  far-fetched?  Not  really.  The  issue  with 
the  subway  incident  involved  the  movement  of  air;  trains  provide  90  percen  t  of  subway 
ventilation.  Train  movement  could  be  beneficial  and  it  could  also  be  catastrophic.  Are 
there  reports  of  additional  attacks?  Are  people  on  the  platform?  If  a  chemical  agent  is 
involved,  has  it  been  identified?  The  chief  needed  more  information. 

Before  firefighters  provide  ventilation  at  a  high-rise  building  fire,  they  often  go  to 
a  floor  above  the  fire  and  open  windows,  trying  to  re-create  the  same  conditions  units 
will  experience  on  the  fire  floor  should  they  decide  to  ventilate  the  fire  floor.  The 
firefighter  is  determining  the  benefits  of  ventilation.  If  a  strong  wind  blows  into  the 
building,  ventilation  right  now  is  not  a  good  idea.  The  fire  fighter  required  additional 
information,  not  unlike  the  chief  operating  at  the  subway  incident. 


This  example  epitomizes  how  experts  use  domain-specific  experience  from  a 
different  scenario  to  create  JIT  mental  models  while  employing  mental  simulation  to 
puzzle  out  a  COA.  The  chief  was  no  expert  regarding  subway  incidents,  yet  he  acted 
expertly.  This  chief  problem  solved  by  comparing  the  workings  of  a  subway  system 
encompassing  a  myriad  of  underground  tunnels  and  a  high-rise  building  that  scraps  the 
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sky.  By  creating  this  analogy,  he  decided  that  the  effects  of  a  train  moving  air,  much  like 
the  wind  blowing  into  the  building,  was  not  beneficial  to  the  operation.  Imagine  the 
benefits  of  getting  a  novice  to  think  at  this  level. 

The  New  York  City  Fire  Department,  along  with  many  other  first-responder 
agencies  is  comprised  of  individuals  with  these  very  talents.  They  are  the  experts.  How 
long  they  remain  as  members  of  their  respective  departments  in  another  matter.  While 
they  still  are,  we  need  to  learn  from  them. 


A  well-known  golf  instructor  commenting  on  the  art  of  teaching  the  motions  of  his 
golf  swing  to  a  novice,  comments  there  is  little  an  expert  can  do  regarding  teaching  a 
swing.  The  learning  needs  to  be  experiential  and  feedback  based.  This  golf  analogy  goes 
for  SET...  “It ’s  about  learning,  not  about  golf.  ”188 


188  Ayers,  et  at,  “Managing  Risk  through  Scenario  Based  Training.” 
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APPENDIX  A 


Pre-experiment  questionnaire  mean  scores 

In  addition  to  asking  the  three  groups  to  provide  their  years  of  service  with  the 
fire  department  and  current  rank,  the  pre-experiment  questionnaire  also  asked  the  groups 
to  rate  their  experience/knowledge  level  in  five  emergency  response  categories.  On  a 
scale  of  1-5,  with  5  indicating  a  high  level  of  experience,  the  experimental  group  rated 
their  natural  disasters  experience/knowledge  level  with  a  mean  rating  of  1.8,  indicating 
the  least  experience  regarding  the  five  categories.  The  control  groups’  ratings  for  natural 
disasters  were  2.7,  while  the  experts’  was  a  3. 


Groups 

Experts:  (10)  Chiefs  (5)  Captains 

Experimental:  (14)  Lieutenants 

Control:  (5)  Captains  (10)  Lieutenants 

Expert 

(14) 

Experimental 

(14) 

Control 

(15) 

Years  of  service  with  the  Fire  Department 

25 

9 

12 

Years  of  service  in  current  rank 

5 

0 

1 

On  a  scale  of  1-5,  with  1  indicating  little  experience 
and  5  indicating  extensive  experience,  rate  your  lire 
department  knowledge/experience  in  the  following 
response  categories 

Expert 

Experimental 

Control 

Knowledge/experience  with  high-rise  building 
firefighting 

3 

2.4 

2.3 

Knowledge/experience  with  subway  related  incidents, 
(i.e.)  fires/emergencies 

4 

2.8 

1.3 

Knowledge/experience  with  CBRNE  related  incidents 

3 

2 

2.5 

Knowledge/experience  with  hazardous  material 
incidents 

3 

2.1 

2.9 

Knowledge/experience  with  natural  disasters 

3 

1.8 

2.7 
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APPENDIX  B 


Pre  Exercise  SA  /  RPD  Group  Briefing 


Situational  Awareness  (SA):  Simply  stated,  SA  is  the  art  of  being  aware  of  your  surroundings  via  SA 
perception,  comprehension,  and  projection  levels. 

Level  1  SA  Perception: 

(initial  size-up) 

Knowledge  of  surroundings,  recognizing  familiarities  along  with 
inconsistencies. 

Level  2  SA  Comprehension: 

(making  sense  of  surroundings) 

Understanding/comprehending  what  you  perceive  along  with  its 
significance  to  incident  goals  and  objectives 

Level  3  SA  Projection: 

(predicting  what  will  happen  next 
and  take  appropriate  actions) 

Projection  of  future  events  (what  will  you  do  with  the  information), 
governed  by  the  synthesis  of  perception  and  comprehension. 

Recognition-Primed  Decision  Model  (RPD):  The  recognition-primed  decision  model  describes  the  way  a 
decision-maker  is  able  to  recognize  a  situation  and  make  a  decision  by  comparing  current  events  to  stored 
experiences. 

Cue  recognition: 

(Color  of  smoke;  volume  of  fire) 

Identification  and  evaluation  of  incident  cues.  Cues  result  in  patterns; 
patterns  are  compared  to  stored  experiences,  (mental  models) 

Pattern  matching: 

(water  normally  cools  &  extinguishes 
fire) 

“Pattern  matching  refers  to  our  ability  to  detect  typicality  and  notice 
events  that  did  not  happen  and  other  anomalies  that  violate  the 

pattern.”  189 

Mental  models: 

(private  dwelling  attic  fires  often 
originate  in  the  basement) 

Mental  models  explain  the  way  things  work!  They  are  our  stored 
experiences  that  help  us  describe,  explain,  and  predict  situations.  190 

Mental  simulations: 

(mentally  rehearing  the  steps  of  a 
roof  rope  rescue  moments  before 
initiating) 

We  evaluate  a  course  of  action  by  consciously  imagining  what  would 
happen  if  we  carried  it  out.  1 9 1 

1^9  Gary  Klein,  Sources  of  Power,  149. 

190  Kevin  Bums,  ‘‘Mental  Models  and  Normal  Errors,”  (eds)  Berndt  Brehmer,  Raanan  Lipshitz,  Henry 
Montgomery,  How  Professionals  Make  Decisions  (Mahwah,  New  Jersey:  Lawrence  Erlbaum  Associates, 
2005):  17. 

191  Klein,  Power  of  Intuition,  26. 
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•  Cues  let  us  recognize  patterns 

•  Patterns  activate  action  scripts 

•  Action  scripts  are  assessed  through  mental  simulation 

•  Mental  simulation  is  driven  by  mental  models  '  92 


Klein,  Power  of  Intuition,  28. 
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APPENDIX  C 


Renegade  tanker  on  bridge  scenario 

Scenario:  Renegade  tanker  on  bridge  1 

As  the  first  engine  company  (pumper)  officer  to  arrive  at  the  scene  of  a  911-generated 
response  for  a  truck  fire  on  a  suspension  bridge  during  evening  rush  hour,  you  receive  the 
following  information  from  police  stationed  at  the  foot  of  the  bridge: 

•  A  tanker  with  a  capacity  of  5,000  gallons  disregarded  police  instructions  and 
attempted  to  use  the  bridge’s  lower  level  that  is  restricted  to  passenger  traffic 
only.  A  police  vehicle  immediately  pursued  the  tanker. 

•  While  attempting  to  stop  the  tanker  before  reaching  mid-span,  the  police  car 
crashed  into  the  center  divider  and  flipped  over,  while  causing  the  tanker  to 
sideswipe  a  passenger  bus  traveling  in  the  same  direction. 

•  The  bus  swerved  before  coming  to  an  abrupt  stop,  as  the  tanker  crashed  into  the 
side  rail  with  the  driver  of  the  tanker  fleeing  the  scene. 

•  Bridge  police  are  diverting  traffic  on  the  bridge  while  attempting  to  contact  the 
police  officer  involved  in  the  chase. 

Upon  receipt  of  the  preceding  infonnation,  you  establish  command  at  the  bridge. 


Scenario  Decision  Point  #1.1: 

•  In  regards  to  the  preceding  scenario,  prioritize  the  following  list  of  actions. 
Explain  your  first  three  actions 

_ ID  product  leaking  from  tanker 

_ Life  hazards  to  occupants  of  bus  and  police  car 

_ Whereabouts  of  driver  leaving  scene 

_ Protecting  first  responders  /risk  vs.  reward 

_ Communication  w/units  and  dispatcher/  request  additional  alarms/resources 

_ Security  of  scene/  isolation  of  area  against  additional  tankers/explosions  (terrorism) 

_ Prevent  contamination 

_ Notify  additional  agencies 
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Expert  group’s  response  key 


Ranking 

Actions 

1 

Comm,  w/units  and  dispatcher/request  additional  alarms/resources 

2 

Protecting  1st  responders  /  risk  vs.  reward 

3 

Life  hazard  to  occupants  of  bus  and  police  car 

Expert  Feedback: 

Communication  with  units  and  the  fire  dispatcher  was  a  common  thread  among 
expert  decision  makers.  The  experts  overwhelming  believed  that  incoming  units  needed 
to  be  alerted  regarding  the  magnitude  of  the  event.  Such  an  alert  would  be  helpful  to 
responding  units  regarding  possible  staging  locations  while  ensuring  units  don  the  proper 
protection  equipment  (PPE).  Operating  units  also  needed  to  request,  via  the  dispatcher, 
the  response  of  specialized  units  and  resources.  For  example,  a  hazardous  material  unit 
may  be  needed  to  confirm  the  identification  of  the  tanker’s  contents  and  mitigate  the  leak; 
a  foam  unit  may  be  required  to  extinguish  burning  fuel,  and  EMS  to  address  medical 
needs. 

At  an  incident  of  this  type,  the  incident  commanders  must  consider  the  “risk  vs. 
reward”  factor.  Will  rescues  endanger  first  responder  personnel?  Protecting  first 
responders,  preventing  them  from  becoming  part  of  the  problem  is  the  most  important 
consideration  in  the  size-up.  If  first  responders  are  not  safe,  they  cannot  help  those  they 
are  entrusted  to  protect.  For  example,  first  responders  may  be  conducting  operations 
amongst  an  unidentified  leaking  product;  therefore,  they  must  consider  possible  exposure 
to  and  contamination  from  a  hazardous  material  and  take  appropriate  protective  actions. 

The  life  hazards  to  the  police  vehicle  and  occupants  of  the  bus  is  paramount.  An 
initial  size-up  of  the  two  vehicles  will  generate  a  resources  list.  For  example,  ambulatory 
victims  will  be  immediately  removed  while  non-ambulatory  and  those  pinned/trapped 
will  require  special  resources.  The  number  of  injured  bus  passengers  and  status  of  police 
officer  will  determine  the  EMS’s  commitment  of  resources. 
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Scenario:  Renegade  tanker  on  bridge 


1 


Approaching  the  incident,  you  observe  the  following  conditions: 

•  The  engine  compartment/cab  section  of  the  tanker  is  involved  in  fire  while 
exposing  the  passenger  bus 

•  A  steady  flow  of  liquid  coming  from  the  rear  compartment  of  the  tanker 

•  An  occupied  passenger  bus  stopped  about  25  feet  beyond  the  tanker 

•  A  police  vehicle  lying  on  its  side  about  25  feet  in  front  of  the  bus 

•  The  product  has  been  identified  as  #  2  fuel  oil 


Because  bridge  traffic  is  now  at  a  standstill  in  both  directions,  additional  units  will  be 
delayed  in  gaining  access  to  the  immediate  area. 


Scenario  Decision  Point  #  1.2: 

•  What  will  you  address  first?  Prioritize  and  explain  the  following  decision¬ 
making  options  (1,  2,  3,  4): 

_ Police  vehicle  lying  on  its  side 

Fuel  tanker  involved  in  fire 


Liquid  coming  from  the  rear  compartment  of  the  tanker 
Occupied  passenger  bus 


Expert  group’s  response  key 


Ranking 

Decision  making  options 

1 

Fuel  tanker  involved  in  fire 

2 

Occupied  passenger  bus 

3 

Police  vehicle  lying  on  side 
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Expert  Feedback 

Addressing  the  burning  fuel  tanker  was  an  overwhelming  concern  of  most 
experts.  Extinguishing  the  fire  solves  numerous  problems  and  has  the  greatest  impact 
regarding  the  protection  of  life  and  property.  If  total  extinguishment  of  the  fire  cannot  be 
attained  due  to  the  contents  burning,  then  at  least  containment  can  be  achieved.  If  the  fire 
is  not  addressed,  the  incident  can  expand.  Additionally,  because  the  fire  exposes  the 
passenger  bus,  containment  would  assist  the  rescue  effort  while  reducing  panic  among 
bus  passengers. 

The  bus  is  occupied,  and  therefore  a  top  priority.  It  is  severely  exposed  to  fire, 
with  a  good  possibility  of  containing  injured  passengers.  The  police  vehicle  lying  on  its 
side  is  the  furthest  from  the  incident  and  may  possibly  require  a  technical  rescue 
(extraction),  and  has  the  least  impact  upon  the  incident.  Although  the  liquid  leaking  from 
rear  of  tanker  is  a  concern,  most  first  responders  are  not  trained  to  the  level  required  to 
mitigate  the  leak  that  is  both  time  consuming  and  requiring  special  equipment. 
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APPENDIX  D 


Explosion  on  subway  platform  scenario 


Scenario:  Explosion  on  subway  platform 


2 


You  are  a  newly  promoted  lieutenant  working  a  day  tour  in  a  mid-town  engine  company 
(pumper). 


•  At  5:00  pm,  your  company  responds  to  a  91 1-generated  dispatch,  reporting  a  loud 
noise  on  a  subway  platform  within  your  first-alarm  district. 

•  The  response  to  this  alarm  consists  of  (2)  two  engine  companies  and  (1)  one 
ladder  company;  The  Battalion  Chief  normally  assigned  is  operating  at  another 
incident  and  is  unavailable. 

•  Your  company  normally  arrives  second  at  this  location,  however,  while  in  route 
you  receive  notification  that  you  are  now  arriving  first  and  that  other  loud  noises 
have  been  reported  at  subway  platforms  serviced  by  the  same  subway  line. 


Scenario  Decision  Point  #  2.1: 


•  Does  the  receipt  of  additional  information  influence  your  decision-making? 
Yes /No.  Explain. 


Group 

Yes  /  No 

Percentage 

Experts 

Yes 

100  % 

Expert  Feedback 

The  receipt  of  additional  infonnation  influenced  experts’  decision  making  in  the 
following  ways:  First,  an  isolated  incident  on  a  subway  platform  could  be  the  result  of  an 
electrical  fire,  derailed  train  car,  or  possibly  a  trash  can  fire,  just  to  name  a  few.  While 
these  events  are  not  everyday  occurrences,  they  do  happen  from  time  to  time  and 
typically  are  isolated  events.  Multiple  incidents,  however,  on  the  same  platform  or  at 
more  than  one  location  and  you  begin  to  recognize  a  pattern.  Given  the  location  (mass 
transit  platfonn)  and  the  history  of  past  attacks  on  similar  locations  (Madrid  and  Tokyo), 
first  responders  need  to  think  terrorism  or  premeditated  planning.  For  example,  several 
loud  noises  equate  to  explosions  and  explosions  equate  to  terrorism.  Additional  reports  of 
explosions  confirm  an  ongoing  event  and  verify  a  consistency  in  information  received 
while  lending  credibility  to  earlier  reports  whether  terrorist  related  or  not. 
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Second,  your  unit’s  responsibilities  change  because  of  additional  information. 
Because  you  will  now  arrive  first  on  the  scene,  you  will  assume  the  role  of  incident 
commander  (until  relieved  by  a  higher  rank)  and  be  responsible  for  transmitting  a 
preliminary  report  regarding  initial  size  up.  Depending  on  rank  structure  and  arrival  order 
of  responding  units,  your  initial  role/responsibilities  may  be  more  strategically  (directing 
operations)  and  less  tactical. 

Scenario  Decision  Point  #  2.2: 

•  Prioritize  the  following  concerns.  Explain  your  first  three  concerns. 

_ Evaluate  status  of  victims  and  evacuation  in  progress 

_ Establish  communication  with  the  transit  authority  for  additional  infonnation 

_ Recognize  Incident  Commander  responsibilities 

_ Conduct  environmental  size  up  (smoke,  fire,  readings) 

_ Adequate  resources  on  scene 


Expert  group’s  response  key 


Ranking 

Concerns 

1 

Recognize  Incident  Commander  responsibilities 

2 

Conduct  environmental  size  up  (smoke,  fire,  readings) 

3 

Evaluate  status  of  victims  and  evacuation  in  progress 

Expert  Feedback 

The  first  officer  assuming  command  at  an  incident  of  this  type  needs  to  recognize 
that  early  establishment  of  command  and  control  is  imperative  for  the  safety  of  operating 
personnel.  Standard  firefighting  operating  procedures,  often  predicated  on  an  aggressive 
attack  of  the  fire  to  protect  life  and  property  may  not  be  applicable  here.  Instead,  more 
time  than  usual  is  required  in  initial  size-up.  While  your  unit’s  initial  duties  may  have 
included  the  stretching  of  hose  lines  or  identification  of  the  hazard,  you  are  now 
responsible  for  the  overall  management  of  the  incident. 

As  the  incident  commander,  you  need  to  conduct  a  size  up,  provide  direction 
(leadership),  and  prioritize  actions  (what  needs  to  be  done  first).  Two  primary  concerns 
include  unit  deployment  and  the  transmission  of  preliminary  reports  to  notifying  higher 
command  of  a  developing  situation,  while  alerting  incoming  units  regarding  the 
magnitude  of  the  event. 
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An  environmental  size-up  regarding  smoke,  fire  and  monitoring  of  the  immediate 
vicinity  for  hazardous  material  release  will  determine  future  actions.  At  an  incident  of 
this  type,  units  cannot  continue  operations  without  additional  information.  There  are  too 
many  unknowns.  For  example,  reports  of  loud  noises  on  a  subway  platfonn  are  not 
unusual;  however,  they  do  need  to  be  investigated.  A  visual  size-up  of  the  subway  air 
vents  at  street  level  may  indicate  conditions  down  on  the  subway  platfonn.  Where  there 
is  smoke  there  is  fire!  Units  need  to  create  patterns  from  cues,  reports  received,  and 
expectations  developed  from  past  events. 

Life  is  a  #1  priority.  At  an  incident  of  this  type,  the  incident  commander  must 
evaluate  the  status  of  victims  and  evacuation  in  progress.  Signs  and  symptoms  of 
evacuating  passengers  provide  information  regarding  the  type  of  event.  What  symptoms 
do  victims  normally  exhibit  when  evacuating  a  fire  related  environment?  Are  the 
passengers  exhibiting  signs/symptoms  that  would  normally  be  expected,  or  are  they 
displaying  signs  and  symptoms  that  are  atypical?  The  evacuation  status  also  provides 
valuable  information  regarding  first  responder  commitment.  For  example,  our  mental 
models  /  expectations  regarding  rush  hour  at  a  subway  platfonn  tells  us  to  expect  a  large 
number  of  passengers  using  the  subway  station.  Therefore,  while  responding,  we 
anticipate  or  expect  to  find  a  mass  exodus  up  the  subway  platform  stairs  to  the  street 
above. 
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Scenario:  Explosion  on  subway  platform 


2 


Upon  arrival  at  the  street-level  entrance  to  the  subway  platform,  you  observe  several 
people  with  minor  injuries  complaining  of  breathing  difficulties;  there  is  a  light  to 
medium  smoke  condition  coming  from  the  subway  vents. 


•  On-scene  transportation  personnel  inform  you  that  there  have  been  a  total  of  three 
loud  noises,  “The  first  noise  occurred  immediately  after  the  train  left  the  station 
and  shortly  after  two  subsequent  noises  were  heard — first  one  then  the  other.” 

•  A  surveillance  camera  located  at  the  far  end  of  the  station  reveals  two  people 
overcome  on  the  platform  as  several  small  fires  burn.  Numerous  backpacks  and 
bags  are  scattered  throughout. 

•  Presently  there  is  no  one  exiting  the  subway  via  the  street  entrance  where  you  are 
located.  It  is  now  5:15  pm  and  the  remaining  assigned  units  are  arriving. 


Scenario  Decision  Point  #  2.3: 


•  Are  you  committing  personnel  to  the  subway  platform?  Yes  /  No.  Explain. 


Group 

Yes 

No 

Experts 

60% 

40% 

Expert  Feedback 

Sixty  percent  of  experts  committed  personnel  to  the  platform.  The  common  thread 
amongst  the  experts  was  the  now  confirmed  life  hazard.  The  signs/symptoms  that 
evacuating  passengers  displayed  were  also  instrumental  in  the  IC  decision  to  commit 
units  to  the  platform.  For  example,  people  complaining  of  breathing  difficulties  while 
serious,  is  a  common  fire  related  symptom  (mental  models  from  experience).  The  smoke 
could  be  the  result  of  a  malicious  trash  can  fire,  with  the  noise  being  the  sound  of  a 
heated  aerosol  container  exploding.  A  majority  of  the  experts  believed  that  a  limited 
commitment  of  resources  outfitted  with  full  protective  equipment  to  rescue  victims, 
extinguish  fires  and  conduct  a  reconnaissance  of  the  immediate  area  was  a  justifiable  risk 
considering  the  rewards. 

Conversely,  forty  percent  of  the  experts  decided  against  committing  personnel  to 
the  platfonn  without  additional  infonnation.  The  common  thread  amongst  this  group  of 
experts  was  that  further  investigation  of  the  scene  was  required,  identifying  hazards 
before  committing  personnel.  This  incident  could  involve  an  improvised  explosive  device 
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with  possible  chemical  or  nerve  gas  components.  Experts  made  this  observation  by 
considering  the  history  of  past  incidents  (sarin  gas  attack  on  Tokyo  subway  system).  The 
limited  evacuation  of  subway  platform  does  not  match  patterns  and  expectations  for  rush- 
hour  time  of  day  and  reinforces  reluctances  to  commit  personnel  to  an  unknown 
environment.  Additionally,  the  use  of  first  responder  radios  could  detonate  secondary 
explosive  devices  designed  to  hann  first  responders  operating  at  the  scene  (Madrid  train 
bombings). 

Scenario  Decision  Point  #  2.4: 


•  At  this  point,  are  you  stopping  train  movement?  Yes  /  No.  Explain. 


Group 

Yes 

Experts 

90% 

Expert  Feedback 

Ninety  percent  of  the  experts  agreed  that  train  movement  should  be  stopped.  The 
moving  trains  provide  subway  tunnel  ventilation;  therefore  stopping  trains  reduces  air 
movement  and  contamination  of  adjoining  stations.  Additional  reasons  for  stopping  trains 
included  the  following:  Isolating  trains  at  previous  station  prevents  trains  from  entering 
the  station  and  exposing  passengers  to  a  possible  secondary  explosive  device;  passengers 
may  have  entered  the  track  area  because  station  was  uninhabitable  or  exited  stranded 
trains  in  tunnel  between  stations;  the  incident  commander  may  need  to  commit  units  to 
the  track  area  and  confirmation  of  power  removal  takes  time. 
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Scenario:  Explosion  on  subway  platform 


2 


From  an  adjacent  subway  entrance  that  serves  the  same  platform  but  not  within  your  line 
of  sight,  units  report  that  they  are  operating  at  street  level  assisting  in  a  mass  evacuation 
of  subway  passengers. 

•  There  are  some  minor  injuries  and  complaints  of  difficulty  with  breathing, 
however  for  the  most  part  passengers  appear  to  be  ambulatory. 

•  Evacuating  passengers  inform  assisting  units  that  there  was  a  loud  noise  just  as 
the  train  pulled  out  of  the  station  and  then  everything  went  black. 


Scenario  Decision  Point  #  2.5: 

•  Does  the  receipt  of  this  information  change  your  current  decision-making 
(committing  personnel  to  platform  /  train  movement)?  Yes  /  No.  Explain. 


Group 

No 

Experts 

70% 

Expert  Feedback 

Seventy  percent  of  the  experts  stayed  with  initial  decision-making.  Reasons  for 
staying  with  initial  plans  include  that  the  new  information  (regardless  of  their  prior 
decisions  to  commit  or  not  commit  personnel  to  platform  or  stop  trains)  does  not 
significantly  influence  their  strategies  regarding  decision  to  commit  or  not  commit  units 
to  platform  and  stopping  train  movement.  For  those  who  decided  to  commit  personnel  to 
platform  the  new  infonnation  reinforces  the  initial  objectives,  namely  that  victims  are 
endangered  and  need  to  be  rescued.  Also,  the  new  information  is  consistent  with  their 
original  assessment  and  expectations  of  explosive  situation  and  decision  to  commit  has 
already  been  initiated.  For  the  experts  deciding  against  the  commitment  of  personnel,  the 
situation  still  remained  undefined  and  a  good  assessment  needs  to  be  made  before 
committing  personnel. 

Scenario  Decision  Point  #  2.5: 

•  Prioritize  the  following  actions.  Explain  your  first  three  actions. 

_ Ensure  adequate  resources  on  hand 

_ Transmit  information  to  dispatch  (progress  report) 

_ Guard  against  secondary  devices/terrorism 

_ Need  additional  infonnation  from  TA 

_ Consider  gross/mass  decontamination  of  passengers 

_ Limited  search  of  subway  w/PPE  ensuring  responder  safety 
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Monitor  immediate  environment 


Expert  group’s  response  key 


Ranking 

Actions 

1 

Limited  search  of  subway  w/PPE  ensuring  responder  safety 

2 

Guard  against  secondary  devices/terrorism 

3 

Transmit  information  to  dispatch  (progress  report) 

Expert  Feedback 

Because  life  is  a  #  1  priority,  a  risk  vs.  reward  evaluation  warrants  a  limited 
search  of  subway  platform.  Adhering  to  PPE  recommendations  allows  first  responders  to 
achieve  goals  of  rescue  and  reconnaissance  while  at  the  same  time  retaining  a  level  of 
safety. 

Other  than  a  visual  inspection,  initial  units  do  not  have  the  capability  to  detect  the 
presence  of  an  explosive  device.  With  this  in  mind,  units  need  to  guard  against  secondary 
devices  designed  to  kill  first  responders,  while  recognizing  their  limits  when  operating  at 
the  scene  of  initial  explosion.  If  units  consider  this  incident  to  be  terrorist  related,  they 
must  recognize  that  the  use  of  secondary  explosives  devices  is  a  pattern  of  past  terrorist 
attacks  (Madrid)  and  guard  against  complacency  and  request  additional  resources. 

Special  first  responder  resources  are  required  at  this  incident.  Their  use  should  be 
recognized  and  requested  as  early  as  possible  in  the  incident.  For  example,  victims  on  the 
platform  and  evacuating  passengers  requiring  medical  attention  will  require  a  greater 
commitment  of  EMS  resources  besides  those  normally  assigned  on  the  initial  response. 
The  immediate  environment  must  be  monitored  by  special  equipment  not  normally 
assigned  to  first  responder  units.  Lastly,  higher  levels  of  command  need  a  size  up-this 
incident  is  part  of  larger  problem 
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Scenario:  Explosion  on  subway  platform 


2 


A  few  minutes  later,  several  evacuated  passengers  are  exhibiting  signs  of  weakness  and 
confusion  and  complaining  of  headaches,  dizziness,  and  nausea.  Three  victims  become 
incapacitated  and  require  immediate  medical  attention. 


Scenario  Decision  Point  #  2.6: 

•  On  a  scale  of  1-5,  with  1  being  very  low  and  5  being  very  high,  how  confident 
are  you  that  this  incident  is  terrorist  related?  Explain. 


Group 

Scored  a  4  or  higher 

Expert 

70% 

Expert  Feedback 

Seventy  percent  of  the  experts  scored  a  4  or  greater  regarding  their  level  of 
confidence  that  this  incident  is  terrorist  related.  Location,  multiple  explosions,  and 
medical  symptoms  all  point  to  terrorism.  Similarities  can  be  drawn  between  this  attack 
and  past  attack  on  comparable  locations  (Madrid  train  bombing).  For  example,  terrorist 
groups  are  known  for  attacks  on  critical  infrastructure  (transit)  along  with  the  time  of  day 
(rush-hour)  and  the  large  number  of  passengers.  Reports  from  evacuating  passengers  that 
an  explosion  occurred  immediately  after  the  train  left  the  station  coupled  with  delayed 
physical  symptoms  and  victim  incapacitation  are  indicative  of  a  chemical  type  attack 
reinforcing  beliefs  that  this  incident  is  terrorism  related. 

Scenario  Decision  Point  #  2.7: 

•  If  this  incident  is  terrorist  related,  what  type  of  attack  might  the  scenario 
information  indicate? 


Type  of  attack 

Chemical 

Explosive 

Dirty  bomb 

Incendiary 

Experts 

7  (50%) 

4 

2 

1 

Scenario  Decision  Point  #  2.8: 

•  Prioritize  the  following  decision-making  actions.  Explain  your  first  three 
actions: 

_ Safety  full  PPE 

_ Request  additional  resources  for  decontamination 

_  Make  necessary  notifications 
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Ensure  passenger  evacuation  continues 
Isolate  and  confine  access  to  area 
Extinguish  fires  burning 
Triage  of  victims 

Re-examine  monitoring  of  environment 
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Expert  group’s  response  key 


Ranking 

Actions 

1 

Safety  full  PPE 

2 

Make  necessary  notifications 

3 

Isolate  and  confine  access  to  area 

Expert  Feedback: 

Experts  agreed  that  in  addition  to  ensuring  that  all  members  adhere  to  PPE 
recommendation  and  notifications  regarding  incident  dynamics  continue  to  be 
transmitted,  the  immediate  area  must  be  isolated.  The  latest  information  regarding  several 
evacuated  passengers  exhibiting  signs  of  weakness,  confusion,  and  complaining  of 
headaches,  dizziness,  and  nausea,  along  with  victims  becoming  incapacitated  and 
requiring  immediate  medical  attention  necessitates  that  the  area  be  sealed.  Intrusion  from 
the  exterior  and  contaimnent  of  those  contaminated  /  injured  until  further  evaluation  in 
the  form  of  decontamination  and  triage  can  be  administered  is  paramount.  If  isolation 
measures  are  not  taken,  the  incident  has  the  capability  of  expending  beyond  immediate 
area. 
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APPENDIX  E 


Radiological  response  scenario 


Scenario:  Radiological  response 

3 

A  highly  publicized  and  controversial  meeting  scheduled  for  the  United  Nations  builc 

ling 

has  resulting  in  several  credible  threats  from  dissent  groups  vowing  to  take  unspecified 
action  if  the  meetings  are  not  canceled. 

•  Due  to  the  seriousness  of  these  threats,  along  with  the  importance  of  these 
meetings  taking  place,  all  affected  first  responders  have  increased  their  awareness 
levels.  In  addition  to  their  normal  security  precautions,  the  police  are  spot 
searching  vans/trucks  entering  Manhattan,  while  fire  department  units  have  been 
reminded  of  the  importance  to  activate  and  monitor  all  CBRNE -related  equipment 
while  out  of  quarters. 

•  Although  all  threats  are  taken  seriously,  precautions  of  this  nature  are  not  unusual 
for  units  that  work  in  this  area.  More  times  than  not,  these  threats  are 
unsubstantiated,  the  awareness  level  is  reduced,  and  units  go  back  to  business  as 
usual. 

•  Throughout  the  first  two  days  of  the  United  Nations  meetings,  no  unusual 
occurrences  have  been  reported;  however,  your  unit’s  false-alarm  rate  during  this 
two-day  period  has  increased  by  more  than  100  percent. 


Scenario  Decision  Point  #3.1: 

•  Given  the  following  information,  how  do  you  prepare  for  the  your  upcoming 
tour  of  duty  on  the  final  day  of  the  United  Nation’s  meeting?  Prioritize  the 
following  actions.  Explain  your  first  three  actions. 

_ Check  pass  along  book  regarding  responses  from  previous  tours 

_ Remind  members  to  guard  against  complacency  due  to  the  unusually  high  false 

alarm  rate 

_ Ensure  all  equipment  is  operational  and  members  are  knowledgeable  with 

equipment  use 

_ Contact  surrounding  companies  for  additional  information 

_ Contact  Police  Department  for  update 

_ Prepare  Drill  on  CBRNE  type  attack 

_ Conduct  briefing  stressing  the  type  of  threats  units  could  encounter 
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Expert  group’s  response  key 


Ranking 

Actions 

1 

Ensure  all  equipment  is  operational  and  members  are  knowledgeable  with 
equipment  use 

2 

Check  pass  along  book  regarding  responses  from  previous  tours 

3 

Remind  members  to  guard  against  complacency  due  to  the  unusually  high  false 
alarm  rate 

Expert  Feedback 

The  experts  agreed  that  ensuring  all  equipment  is  operational  and  members  are 
knowledgeable  with  equipment  use  was  a  top  action.  Understanding  that  predicting  what 
might  happen  on  any  given  tour  is  impossible  and  that  other  than  signs  /  symptoms,  the 
only  way  to  detect  the  presence  of  radiation  is  through  the  monitoring  of  the 
environment,  working  equipment  and  knowledge  of  its  use  is  imperative. 

The  placing  of  equipment  in  or  out  of  service  is  just  one  of  the  many  entries  that 
may  be  found  in  a  unit’s  pass  along  book.  Experts  believed  that  one  of  the  first  steps  in 
preparation  for  the  upcoming  tour  was  to  check  unit’s  pass  along  book  entries  recorded 
since  their  last  tour  of  duty.  Entries  will  capture  any  unusual  events  that  have  occurred  on 
prior  tours  creating  patterns  regarding  specific  types  of  response.  For  example,  a  unit  may 
record  notifying  the  next  level  of  command  regarding  the  unusually  high  false  alarm 
response  rate  to  a  specific  location. 

Pattern  of  high  false  alarm  rate  with  no  explanation  reinforces  the  need  to  remind 
members  to  guard  against  complacency  due  to  the  unusually  high  false  alarm  rate.  For 
example,  terrorist  may  be  monitoring  response  patterns  to  certain  locations,  waiting  for 
apathy  to  sets  in,  resulting  in  firefighter  complacency  followed  by  terrorist  strike 
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Scenario:  Radiological  response 


3 


At  11:00  am  on  the  third  and  final  day  of  the  meetings,  while  responding  first  due  to  a 
street-box  alann  that  requires  you  to  pass  within  one  block  of  the  United  Nations 
building,  your  unit  receives  a  reading  of  (1  mr/hr)  on  their  radiological  detector.  This  is 
the  first  radiological  reading  units  have  received  since  the  meetings  commenced.  The 
alarm  location  you  are  responding  to  has  been  one  of  your  recent  chronic  false  alarms 
boxes. 


Scenario  Decision  Point  #  3.2: 

•  Do  you  stop  to  investigate  or  proceed  to  original  response  location?  Stop  / 
Proceed.  Explain. 


Groups 

Proceed 

Stop 

Experts 

60% 

40% 

Expert  Feedback 

Sixty  percent  of  the  experts  believed  that  unit  must  continue  to  original  response 
location.  No  known  life  hazards  along  with  radiological  readings  too  low  or  not  unusually 
high  (possibly  background  readings)  were  among  reasons  for  proceeding  to  original 
location  and  not  stopping  to  investigate.  Units  did  notified  the  dispatcher  regarding 
readings  (location/rate)  while  passing  along  information  to  units  responding  to  original 
alarm  and  requested  the  response  of  nearest  available  unit  or  hazardous  material  unit  with 
sophisticated  equipment  to  verify  readings. 

Forty  percent  of  the  experts  believed  readings  were  high  enough  to  warrant 
investigation  for  verification  and  that  some  type  of  radiological  event  has  taken  place.  A 
reading  of  1  mr/hr  on  apparatus  may  indicate  higher  reading  closer  to  source  and  units 
may  be  approaching  an  area  with  higher  readings.  Unit  notified  dispatcher  of  readings 
and  that  they  are  stopping  to  investigate  while  requesting  an  additional  unit  to  verify 
readings  with  a  second  meter.  Additional  units  can  continue  to  original  alarm  location 
that  has  been  chronic  false  alarm  box. 
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Scenario:  Radiological  response 


3 


A  few  moments  later  your  unit  is  redirected  to  a  reported  traffic  accident  one  block  north 
of  the  United  Nations.  While  entering  the  block  of  the  reported  location,  your  apparatus 
narrowly  avoids  a  head-on  collision  with  a  car  driving  against  oncoming  traffic.  The  car 
comes  to  an  abrupt  halt  with  two  men  exiting  and  running  away  from  the  scene.  You 
continue  to  monitor  your  readings  that  have  increased  and  are  now  holding  steady  at 
(lOmr/hr).  Sizing-up  the  accident  scene  you  observe  the  following: 


•  A  traffic  accident  involving  a  van  and  a  small  bus  transporting  about  10 
passengers.  Writings  on  the  side  and  front  of  the  bus  are  in  English  and  Hebrew 

•  The  van  has  crashed  into  the  side  of  the  bus  pinning  the  bus  driver  to  his  seat  and 
blocking  the  only  exit  door  on  the  bus 

•  A  small  fire  bums  under  the  van 

•  You  are  now  receiving  a  reading  of  (5r/hr)  on  your  radiological  detector 


Scenario  Decision  Point  #  3.3: 

•  Considering  the  “risk  vs.  reward”  commitment  of  resources,  what  will  you 
address  first?  Prioritize  the  following  decision-making  options.  Explain  your 
first  three  decision-making  options. 

_ Address  life  hazard  on  the  bus 

_ Transmit  size-up  and  resource  request 

_ Extinguish  the  fire 

_ Identify  hazards  and  monitor  environment 

_ Ensure  all  members  wear  full  PPE 

_ Isolate  the  area  due  to  confirmed  radiological  threat  (hot  zone) 


Expert  group  response  key: 


Ranking 

Options 

1 

Ensure  all  members  wear  full  PPE 

2 

Transmit  size-up  and  resource  request 

3 

Address  life  hazard  on  the  bus  /Isolate  the  area 

Expert  Feedback 

Ensuring  that  all  personnel  wear  full  PPE  protects  firefighters  so  they  can  protect 
life.  Keeping  firefighters  from  becoming  victims  themselves  is  an  incident  commander’s 
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primary  concern.  At  this  incident,  the  inhalation  of  contaminants  is  an  immediate  threat 
to  firefighters.  The  verified  radiological  readings  (5  r/h)  in  immediate  area  of  bus  and  van 
dictate  that  firefighter  don  PPE,  providing  sufficient  protection  for  short-term  exposure 
(alpha  /  beta). 

This  incident  is  beyond  the  capabilities  of  a  single  unit  response,  necessitating 
that  units  transmit  a  size-up  alerting  incoming  units  regarding  the  magnitude  of  the 
incident  while  requesting  additional  resource.  A  size  up  alerts  higher  command  to 
escalating  incident  while  providing  valuable  information  to  responding  units.  For 
example,  incoming  units  need  to  know  location  of  contamination  controls  zones.  A 
resource  request  gets  specialized  units  and  equipment  on  the  scene  (Haz-Mat 
unit/decontamination)  or  at  least  on  the  road,  additional  manpower  affords  the  incident 
commander  the  ability  to  rotate  personnel. 

Isolation  of  immediate  area  is  required  pending  further  size  up  while  protecting 
members  from  additional  exposure.  Reading  of  5  r/h  confirms  a  radiological  threat  and 
necessitates  the  evacuation  of  immediate  area  and  establishment  of  contamination  control 
zones  to  identify  areas  requiring  special  equipment  and  personal  protection  levels.  Life 
hazard  on  the  bus  is  a  #1  priority.  There  is  a  medium  risk  (exposure)  for  high  reward 
(life).  Units  can  make  rescues,  evacuating  ambulatory  passengers  first  and  rotating 
personnel  for  technical  rescues  limiting  exposure  to  radiation. 


Scenario:  Radiological  response 


3 


While  sizing-up  the  incident,  your  chauffeur  informs  you  that  additional  units  are  being 
assigned  to  this  location  and  that  similar  incidents  are  being  reported  throughout  the  area. 


Scenario  Decision  Point  #  3.4: 

•  Does  this  new  information  change  your  decision  making  process  regarding 
your  unit’s  commitment?  Yes  /  No.  Explain. 


Group 

No 

Experts 

80% 
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Expert  Feedback 

Eighty  percent  of  the  experts  were  already  treating  this  as  a  terrorist  related 
incident,  taking  the  necessary  precautions  and  actions  to  ensure  firefighter  safety  (PPE, 
isolation,  notifications).  They  were  committed  to  incident  and  recognized  immediate 
problems  i.e.,  life,  fire,  hazardous  materials,  etc... The  additional  information  confirms 
assumptions  regarding  terrorism  and  heightens  the  likelihood  that  this  is  a  terrorist  related 
event,  however  it  does  not  change  decision  making  process  —  objectives  regarding  the 
incident  remains  the  same  ,  a  radiological  attack  w/victims. 
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APPENDIX  F 


Earthquake  scenario 


Scenario:  Earthquake 


4 


It  is  January  2,  2008,  and  you  are  working  a  day-tour  in  Ladder  Company  999  located  in 
Queens,  New  York. 

•  Ladder  999 ’s  response  area  is  mainly  residents  consisting  of  private  dwelling 
wood-frame  buildings  that  are  two  to  three  stories  in  height.  Although  the 
neighborhood  is  composed  of  several  ethnicities,  most  of  the  local  residents  are 
Hispanic. 

•  While  in  the  process  of  completing  your  0900  roll  call  and  familiarizing  yourself 
with  apparatus  tools  and  equipment,  the  ground  beneath  you  begins  to  shake 
causing  you  to  momentarily  lose  your  balance,  as  breaking  glasses/dishes  can  be 
heard  from  the  kitchen  area. 

•  The  shaking  lasts  no  more  than  twenty  seconds;  however,  it  is  immediately 
followed  by  two  additional  tremors  that  are  shorter  in  length  and  less  severe.  The 
distinguishable  error/failure  tone  emitting  from  the  house  watch  computer  is 
drowned-out  by  the  high-pitched  sound  of  car  and  house  alarms  heard  up  and 
down  the  street  of  the  firehouse. 


Decision  Point  #  4.1: 

•  From  the  information  received  up  to  this  point,  prioritize  the  following  list  of 
decision-making  actions  and  explain  your  first  three  actions. 

_ Monitor  media  for  additional  information  (television/radio) 

_ Establish  communications  w/dispatcher 

_ Prepare  for  emergency  type  responses  (gas  and  water  mains,  downed  trees/power 

lines) 

_ Assessment  of  members/firehouse 

_ Prepare  for  verbal  alarms  received  at  firehouse 

_ Establish  communications  with  immediate  supervisor 


Expert  group’s  response  key 


Ranking 

Action 

1 

Assessment  of  members/firehouse 

2 

Establish  communications  w/dispatcher 

3 

Establish  communications  with  immediate  supervisor 
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Expert  Feedback 

The  condition  of  personnel  and  structural  stability  of  firehouse  is  the  primary 
concern.  Firefighters  cannot  help  civilians  if  they  are  unable  to  respond  because  of 
personal  injuries  or  structural  damage  to  firehouse  (utilities,  apparatus  doors,  apparatus 
floor,  and  apparatus).  Understanding  that  few  fire  officers  are  structural  engineers,  a 
visual  inspection  of  the  firehouse  building  needs  to  be  conducting  nonetheless.  Among 
the  questions  that  need  to  be  answered  include:  Can  the  building  withstand  a  similar 
tremor?  Does  unit  need  to  evacuate  due  to  structural  damage  to  firehouse? 

By  contacting  dispatcher,  unit  establishing  a  reliable  communication  link  while 
notifying  the  dispatcher  of  unit’s  availability  for  response.  For  example,  if  units  cannot 
open  the  firehouse  apparatus  doors,  they  cannot  respond.  Units  can  also  determine  extent 
of  earthquake  damage  outside  of  their  immediate  response  area.  Is  it  widespread  or 
limited  in  scope?  This  is  important  because  unit  will  know  how  much  support  it  can 
expect  from  surrounding  units.  Communication  with  dispatcher  will  also  help  to  appraise 
next  actions. 

In  addition  to  establishing  communication  with  dispatcher,  units  also  need  to 
contact  the  next  level  of  command.  The  chief  will  be  gathering  additional  information  to 
determine  safest  response  matrix,  therefore  they  need  to  know  unit  status  and  availability. 
For  example,  if  a  unit  is  in-service  but  responding  undennanned,  the  chiefs  priorities 
may  need  to  change  regarding  the  protection  of  life  and  property,  they  may  not  be  able  to 
perform  both. 
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Scenario:  Earthquake 


4 


You  find  out  that  at  approximately  8:58  am,  the  New  York  City  area  experienced  an 
earthquake  with  a  reported  5.5  to  6.0-magnitude  range.  Tremors  have  been  reported  for 
up  to  100  miles  of  the  epicenter  located  just  north  of  the  city  with  additional  aftershocks 
reported  throughout  your  area. 


•  At  9:10am,  you  are  dispatched  via  department  radio  to  a  reported  explosion  and 
tire  at  a  gas  station  located  within  two  blocks  of  the  firehouse. 

•  Before  leaving  quarters,  you  receive  information  regarding  a  possible  robbery  in 
progress  at  your  response  location.  Additional  fire  units  have  been  assigned  to  the 
alarm  and  the  police  are  in  route. 

•  While  responding  you  are  flagged  down  by  neighbors  telling  you  that  their  child 
is  hurt  and  needs  medical  care.  Although  you  are  able  to  understand  bits  and 
pieces  of  the  conversation,  the  language  barrier  prevents  you  from  obtaining  any 
additional  information  regarding  the  extent  and  circumstances  surrounding  the 
child’s  injuries. 


Scenario  Decision  Point  #  4.2 

•  Does  this  new  information  regarding  a  possible  robbery  in  progress  and 
verbal  alarm  for  injured  child  change  your  response  procedures?  Yes  /  No. 
Explain. 


Expert  group’s  response  key 


Group 

Yes  /  No 

Percentage 

Experts 

Yes 

100  % 

Expert  Feedback 

Experts  overwhelming  agreed  that  the  child  was  an  immediate  life  threat  and  must 
be  addressed.  Personal  contact  with  civilian  (verbal  alarm)  confirms  an  incident  that  may 
require  prompt  action  if  life  threaten.  Prioritizing  this  incident  will  influence  next 
decision  regarding  unit  remaining  on  the  scene  or  proceeding  to  original  alarm.  If  injuries 
are  not  life  threatening,  the  officer  may  consider  leaving  one  firefighter  with  child  until 
arrival  of  EMS. 
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Scenario:  Earthquake 

4 

You  stop  and  discover  the  child’s  injuries  to  be  minor  and  already  under  the  care  of  El 
In  the  meantime,  the  dispatcher  has  received  several  calls  reporting  smoke  and  a  st 
collapse  in  the  same  vicinity  of  the  gas  station.  The  police  are  also  reporting  widespi 
looting  throughout  the  area. 

MS. 

reet 

*ead 

Scenario  Decision  Pt  #  4.3: 

•  Place  a  check  next  to  the  following  decision-making  actions/options  you  agree 
with  and  rank  which  actions/options  are  most  important.  Explain  your  first 
three  actions. 

_ Proceed  to  original  alarm  but  wait  for  police  before  investigating 

_ Advise  chauffeur  to  slow  down  and  proceed  with  caution 

_ Inform  members  of  additional  information  regarding  reports  of  a  robbery  inprogress 

and  street  conditions 

_ Use  of  heavy  caliber  streams  from  a  distance 

_ Wait  for  remaining  alann  assignment  before  continuing  into  immediate  area 

_ Consider  hydrant  system  to  be  compromised 

_ Due  to  reports  of  civil  disturbance,  maintain  unit  integrity  when  operating 


Expert  group’s  actions/options  response  key 


Ranking 

Actions  /  Options 

1 

Inform  members  of  additional  information  regarding  reports  of  a  robbery  in 
progress  and  street  conditions 

2 

Advise  chauffeur  to  slow  down  and  proceed  with  caution 

3 

Due  to  reports  of  civil  disturbance,  maintain  unit  integrity  when  operating 

Expert  Feedback 

Upon  arrival  at  the  scene,  operating  units  need  to  be  on  same  page  regarding 
operations.  By  informing  members  of  additional  information  regarding  reports  of  a 
robbery  in  progress  and  street  conditions,  members  will  understand  circumstances  of 
response  before  arrival 

Anticipating  the  infrastructure  to  be  compromised  (traffic  lights,  gas,  electric), 
along  with  the  possibility  of  further  street  collapse  occurring  due  to  weight  of  apparatus 
and  undennined  street  from  water  main  breaks,  necessitates  that  the  officer  and  chauffer 
communicate  regarding  response. 
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Do  not  allow  members  to  become  victims  of  civil  disturbance/robbery.  Operating 
as  a  team  (no  member  operates  alone)  maintains  unit  integrity,  allowing  officers  to  assess 
the  “big  picture”  before  committing  members  to  individual  tasks 


Scenario:  Earthquake 


Because  your  response  route  is  obstructed,  you  have  to  navigate  around  several  displaced 
vehicles  and  assorted  debris. 

•  You  observe  numerous  small  fires  burning  in  the  distance,  but  no  significant 
damage,  other  than  a  few  downed  trees  and  electrical  wires  detected  within  your 
immediate  vicinity. 

•  Your  chauffeur  informs  you  that  this  gas  station  is  situated  in  an  area  comprised 
of  two  ten-story  residential  buildings  and  a  two-story  hospital. 

•  Monitoring  the  department  radio,  you  are  aware  that  fire/EMS  units  are 
overwhelmed  with  medical,  gas  odors,  electrical,  water  pressure  problems, 

_ etc., ...  types  of  emergencies. _ 


Scenario  Decision  Point  #  4.4: 

•  Do  you  anticipate  that  the  hospital  or  residential  buildings  will  be  an 
immediate  concern?  Yes  /  No. 


Group 

Yes  /  No 

Percentage 

Experts 

Yes 

100  % 

Scenario  Decision  Point  #  4.5: 

•  Prioritize  the  following  list  of  decision-making  situations  you  might  encounter. 
(1  being  the  most  important  and  9  the  least).  Explain  your  first  three  decision¬ 
making  situations. 

_ Stuck  elevators  at  both  locations 

_ Exposure  problems  from  possible  falling  glass  (ten-story  buildings) 

_ Extension  of  fire  from  gas  station 

_ Possibility  of  multiple  high-pressure  gas  leaks  at  location 

_ Movement  of  patients  may  become  an  issue  (evacuation  or  shelter  in  place) 

_ Limited  emergency  -type  resources  available  to  assist 

_ Smoke  (hydrocarbons)  from  fire  may  affect  hospital  and  apartment  complex 

_ No  electrical  power  in  area 

_ Request  for  assistance  from  hospital  staff  or  apartment  residents 
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Expert  group’s  response  key 


Ranking 

Decision-making  situations 

1 

Extension  of  fire  from  gas  station 

2 

Possibility  of  multiple  high-pressure  gas  leaks  at  location 

3 

Limited  emergency  type  resources  available  to  assist 

Expert  Feedback 

Extension  of  fire  from  the  gas  station  presents  an  immediate  hazard  with  greatest 
potential  to  threaten  life/property.  Although  units  are  already  overwhelmed,  by 
addressing  the  fire,  they  have  the  opportunity  to  impact  and  influence  the  situation. 
Because  resources  are  limited  and  water  source  questionable,  fire  must  be  contained  and 
prevented  from  extending  (not  necessarily  extinguished).  The  situation  will  get  worse  if 
fire  extends  —  do  not  allow  fire  extension  to  add  to  problem. 

Not  unlike  the  fire,  gas  leaks,  if  not  controlled,  have  the  potential  to  significantly 
impact  the  incident.  The  availability  of  multiple  ignition  sources  throughout  area  (fires, 
downed  electrical  lines),  along  with  the  potential  of  ignited  gas  leaks  under  pressure 
leading  to  a  fast-moving  fire,  necessitates  units  consider  such  actions  as  building 
evacuation  and  gas  dispersion. 

Due  to  limited  availability  of  emergency-type  resources,  the  magnitude  of 
incident  will  overwhelm  resources.  If  the  incident  is  widespread,  numerous  events  will 
have  taken  place,  and  requests  for  additional  resources  will  be  a  priority  and  prevalent 
throughout  city.  The  strain  on  personnel  to  do  many  things  will  be  great.  Units  must 
prepare  to  be  self-sufficient  while  recognizing  their  limits. 


Scenario:  Earthquake 


4 


Upon  arrival,  you  observe  someone  lying  face  down  in  front  of  the  station  with  two  men 
running  from  the  location,  opposite  your  approach. 

•  The  streets  in  and  around  the  station  are  crowded  with  people  either  requesting 
help  or  seeking  information  about  the  initial  earthquake. 

•  Incident  size-up  reveals  gasoline  leaking  from  several  fuel  pumps  with  fuel 
pooling  in  and  around  the  station  and  street. 

•  Gasoline  has  also  been  discovered  coming  up  through  the  ground  in  the  rear  of  the 
station.  This  fuel  is  flowing  down  the  street  adjacent  to  the  curb  line  and  smoke  is 
venting  under  pressure  from  the  front  comice  of  the  station. 
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Scenario  Decision  Point  #  4.6: 

•  Prioritize  your  decision  making.  What  problems  do  you  address  first,  second, 
third...?  Explain  your  first  three  decisions. 

_ Fuel  coming  from  rear  of  station  and  flowing  down  the  street 

_ Person  lying  face  down 

_ Smoke  from  cornice 

_ Fuel  pooling  in  front  of  station 

_ People  requesting  help/information 

_ Lack  of  resources  (foam/EMS) 


Expert  group’s  response  key 


Ranking 

Prioritized  Decision-Making 

1 

Person  lying  face  down 

2 

Fuel  coming  from  rear  of  station  and  flowing  down  the  street 

3 

Smoke  from  cornice 

Expert  Feedback 

Person  lying  face  down  is  a  known  life  hazard  (visual  confirmation)  and 
addressing  life  is  a  #  1  priority  before  property.  Unit  can  make  quick  assessment  and 
determine  viability  of  victim  removal.  The  person  lying  down  may  possibly  be  victim  of 
robbery  and  requiring  immediate  help. 

Fuel  coming  from  rear  of  station  and  flowing  down  the  street  needs  to  be 
controlled.  If  flow  cannot  be  stopped,  units  must  make  an  attempt  to  contain  or  divert 
flow  from  reaching  an  ignition  source.  Leaking  fuel  is  largest  threat  to  life/property,  if 
ignited  it  will  intensify  an  already  dynamic  and  developing  situation. 

Extinguishing  the  fire  eliminates  a  major  problem.  For  example,  smoke  from  the 
fire  has  potential  to  expose  hospital  and  apartment  building.  Investigating  the  source  of 
smoke  will  reveal  fire  intensity,  determining  if  a  quick  stop  of  fire  can  be  accomplished 
or  whether  operating  personnel  should  be  moved  to  a  safer  area 


125 


Scenario:  Earthquake 


4 


Members  conducting  a  perimeter  size-up  report  water  under  pressure  (along  with  dirt  and 
sand)  coming  from  several  large  cracks  in  the  street  that  separates  the  gas  station  from  the 
hospital.  This  condition  also  appears  to  be  undermining  the  sidewalk  that  leads  directly 
into  the  apartment  complex. 


Scenario  Decision  Point  #  4.7: 

•  What  decisions  need  to  be  made  regarding  street  management?  List  your 
concerns  and  explain  your  first  three  concerns. 


Expert  group’s  response  key: 


Ranking 

Concerns 

1 

Position  apparatus  out  of  collapse  area. 

2 

Establish  safe  access  /  egress  from  apartment  building 

3 

Ensure  operating  and  incoming  units  are  aware  of  conditions 

Expert  Feedback 

Due  to  the  structural  integrity  of  the  street,  apparatus  must  be  positioned  outside 
of  a  potential  street  collapse  area.  The  weight  of  apparatus  and  already  undermined 
street/sink  holes  are  conducive  to  further  collapse.  Secondary  access  points  to  and  from 
hospital/apartment  complex  also  need  to  be  identified.  If  hospital/apartment  buildings 
need  to  be  accessed/evacuated,  undermined  street/sidewalks  will  limit  availability  of  safe 
access  and  egress  routes. 

Officers  must  ensure  that  both  operating  and  incoming  units  are  aware  of 
conditions.  Do  not  assume  that  everyone  knows  what  you  know  and  see.  Individuals 
operate  at  different  situational  awareness  levels  and  may  not  share  the  same 
comprehension/understanding  of  the  incident  as  you  do.  In  all  likelihood,  the  stopping  of 
unnecessary  traffic  has  already  been  initiated;  however,  current  conditions  may  require 
that  the  isolation  zone  be  expanded,  limiting  apparatus  and  excessive  pedestrian  traffic  in 
the  area. 
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Scenario:  Earthquake 


4 


At  approximately  9:30  a.m.  you  begin  to  notice  the  ground  shaking  in  your  area.  The 
sidewalk  and  street  has  now  caved  in,  cutting  off  your  access  to  the  apartment  complex 
and  hospital.  There  is  a  distinct  smell  of  natural  gas  throughout  the  area  and  people  are 
beginning  to  exit  the  surrounding  buildings  and  hospital,  reporting  odors  of  gas/fuel. 


Scenario  Decision  Point  #  4.8: 

•  Regarding  the  hospital  and  ten-story  residential  buildings,  a  decision  may 
need  to  be  made  concerning  evacuating  or  sheltering  in  place.  List  those 
points  that  must  be  considered  first  before  any  action  can  be  taken.  Explain 
your  first  three  points 


Expert  group’s  response  key 


Ranking 

Points  to  be  considered 

1 

Can  gas  leaks  be  controlled? 

2 

Available  of  resources 

3 

Are  you  evacuating  people  to  a  safer  location? 

Expert  Feedback 

Can  units  determine  if  reports  of  natural  gas  odors  are  from  high  pressure/low 
pressure  main?  This  information  will  be  invaluable  to  specialized  fire  units  and  utility 
companies  that  have  the  capability  of  shutting  down  leaks  and  can  be  useful  to  these  units 
while  in  route.  If  gas  leaks  are  within  a  hospital  or  surrounding  building,  can  the  leaks  be 
located,  stopped,  or  controlled?  If  leak  can  be  stopped,  conditions  improve  (evacuation 
may  not  be  required).  If  gas  leaks  are  from  an  exterior  source,  shelter  in  place  may  be 
preferred  option 

Availability  of  resources  will  dictate  what  units  can  and  cannot  do.  Does  the 
incident  commander  have  enough  resources  (fire  /  civilians)  to  conduct  an  evacuation? 
Due  to  limited  resources,  can  ambulatory  patients  be  moved  and  non-ambulatory 
sheltered  in  place?  The  incident  commander  must  ascertain  available  resources  in  the 
form  of  hospital  staff  and  civilians. 

Regarding  the  evacuation  or  sheltering  in  place  of  hospital  or  apartment  complex, 
several  questions  must  be  addressed.  Are  apartment  and  hospital  residents  in  immediate 
danger?  Where  will  you  evacuate  people  and  are  you  exposing  them  to  greater  danger 
(street  conditions)  by  evacuating?  How  safe  are  access  and  egress  routes  from  hospital 
and  apartment  buildings? 
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APPENDIX  G 


Experimental  group’s  decision  point  questionnaire 

Upon  the  completion  of  each  scenario,  the  experimental  group  completed  a 
decision  point  questionnaire.  This  questionnaire  was  designed  to  capture  decision-making 
responses  in  real  time;  immediately  after  they  were  made.  A  review  of  this  questionnaire 
revealed  the  SA  /  RPD  model  briefing  components  received  an  above-average  rating 
regarding  usefulness.  Although  there  was  no  pre-experiment  data  to  compare  these 
ratings  against,  the  results  are  nevertheless  encouraging  for  several  reasons.  First,  the 
experimental  group  members  received  a  minimal  SA  /  RPD  model  briefing  regarding  a 
complex  topic,  and  yet  the  group’s  awareness  level  was  above  average.  Second,  the 
impetus  for  identifying  methods  for  improving  first-responder  decision  making  by  raising 
their  situation  awareness  levels  is  strengthened.  The  control  group  did  not  complete  this 
questionnaire 


On  a  scale  of  1-5,  with  1  being  very  low  and  5 
being  very  high,  rate  the  following: 

Tanker 
DP  #  1. 

2 

Subway 
DP  #  2  .3 

Radiological 

DP  #3.  3 

Earthquake 
DP  #  4.  6 

How  difficult  was  this  particular  decisions? 

1  2  3  4  5 

(low)  (high) 

2 

3.4 

2.9 

3.1 

How  much  confidence  do  you  have  with  the 
decisions  you  made? 

1  2  3  4  5 

(low)  (high) 

4.3 

3.2 

3.9 

3.6 

How  much  did  you  rely  on  experience  in  your 
decision-making  process?  (mental  models) 

1  2  3  4  5 

(low)  (high) 

3.1 

2.9 

2.9 

3.2 

How  often  did  you  mentally  rehearse  a  course  of 
actions  prior  to  making  a  decision?  (mental 
simulation ) 

1  2  3  4  5 

(low)  (high) 

4.1 

3.5 

3.6 

3.3 

How  successful  were  you  in  creating  patterns  from 
identified  cues  and  using  those  patterns  to 
recognize  familiarities  and  inconsistencies? 

(cues  +  patterns  =  decisions) 

1  2  3  4  5 

(low)  (high) 

3.7 

3.3 

3.6 

3.5 

Perception:  (initial  size-up) 

3.6 

3.6 

3.7 

3.9 
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Rate  your  ability  to  effectively  size  up  of  the 
incident. 

1  2  3  4  5 

(low)  (high) 

Comprehension:  (making  sense  of 

surroundings) 

Rate  your  ability  to  “make  sense  of  surroundings” 
based  on  initial  size-up,  incident  goals  and 
objectives. 

1  2  3  4  5 

(low)  (high) 

4 

3.7 

3.7 

3.9 

Projection:  (predicting  what  will  happen  next 
and  take  appropriate  actions) 

Rate  your  ability  to  predict  what  will  happen  next 
and  take  appropriate  actions. 

1  2  3  4  5 

(low)  (high) 

3.6 

3.4 

3.5 

3.6 
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Post-experiment  questionnaire 

Both  the  control  and  experimental  groups  completed  a  post-experiment 
questionnaire  designed  to  evaluate  scenario  difficulty,  decision-making  confidence,  and 
influence  of  the  independent  variables.  The  experimental  group  rated  the  influence  of 
expert  feedback  on  their  overall  decision  making  with  a  mean  rating  of  4.1  on  a  scale  of 
1-5  and  the  helpfulness  of  the  SA/RPD  model  theory  briefing  with  a  mean  rating  of  3.4. 
These  above-average  scores,  when  coupled  with  the  experimental  groups’  high  scores 
during  the  actual  earthquake  scenario,  add  credibility  to  research  findings,  showing  that 
expert  feedback  made  a  difference.  The  credibility  of  research  findings  are  further 
reinforced  when  compared  to  the  control  group  that  received  no  feedback  and  scored 
noticeably  lower  on  the  earthquake  scenario  responses. 

When  responding  to  the  post  experiment  questionnaire,  the  control  group 
members  were  unaware  that  the  experimental  group  received  any  additional  treatment. 
Therefore,  their  ratings  were  not  skewed,  biased,  or  pre-disposed  toward  additional 
training.  Yet,  the  group  rated  how  influential  the  receipt  of  expert  feedback  would  have 
been  with  a  mean  rating  of  4.5  on  a  scale  from  1-5.  The  group  also  rated  how  helpful  a 
pre-scenario  decision-making  skills/theory  training  would  have  been  to  their  decision¬ 
making  with  a  mean  rating  of  4.5. 
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On  a  scale  of  1-5,  rate  the  preceding  exercise 

Regarding  decision  making,  how  difficult  were  these 
scenarios? 

Experimental 

Control 

Renegade  Tanker  1  2  3  4  5 

(low)  (high) 

3.4 

2.9 

Subway  1  2  3  4  5 

(low)  (high) 

3.5 

3.7 

Radiological  Attack  1  2  3  4  5 

(low)  (high) 

3.9 

3.5 

Earthquake  1  2  3  4  5 

(low)  (high) 

4.1 

3.6 

How  confident  were  you  with  your  decision  making? 

Experimental 

Control 

Renegade  Tanker  1  2  3  4  5 

(low)  (high) 

3.7 

3.9 

Subway  1  2  3  4  5 

(low)  (high) 

3.4 

3.4 

Radiological  Attack  1  2  3  4  5 

(low)  (high) 

3.1 

3.1 

Earthquake  1  2  3  4  5 

(low)  (high) 

3.1 

3.1 

How  influential  was  the  receipt  of  expert  feedback  after  each 
scenario  decision  point  on  subsequent  decision  making? 

1  2  3  4  5 

(low)  (high) 

4.1 

How  helpful  was  the  SA/RPD  model  theory  briefing  on  your 
decision  making?  (pattern  and  cue  recognition,  mental 
models,  mental  simulation,  SA  levels-perception, 

comprehension,  projection ) 

1  2  3  4  5 

(low)  (high) 

3.4 

How  influential  do  you  believe  the  receipt  of  expert  feedback 
after  each  scenario  decision  point  would  have  been  on 
subsequent  decision  making? 

1  2  3  4  5 

(low)  (high) 

4.5 

How  helpful  do  you  believe  the  pre-scenario  decision-making 
skills/theory  training  would  have  been  to  your  decision 
making? 

1  2  3  4  5 

(low)  (high) 

4.5 
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Earthquake  scenario  score 
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APPENDIX  J 


Earthquake  Scenario  One-way  ANOVA 


Dependent  (Experiment  and  Control  Combined) 


95%  Confidence  Interval  for 
Mean 

N 

Mean 

Std. 

Deviation 

Std. 

Error 

Lower 

Bound 

Upper  Bound 

Minimum 

Maximum 

Experiment 

14 

86.8571 

6.54989 

1.75053 

83.0753 

90.6389 

72.00 

94.00 

Control 

15 

73.6000 

11.59310 

2.99333 

67.1800 

80.0200 

48.00 

100.00 

Total 

29 

80.0000 

11.51397 

2.13809 

75.6203 

84.3797 

48.00 

100.00 

anova  Dependent  (Experiment  and  Control  Combined) 


Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between 

Groups 

1272.686 

1 

1272.686 

14.087 

.001 

Within  Groups 

2439.314 

27 

90.345 

Total 

3712.000 

28 
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